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14.  ABSTRACT 

In  today’s  technology  based  society  the  rapid  proliferation  of  new  machines  and  systems  that  would  have  been  undreamed  of  only  a  few  short 
years  ago  has  become  a  way  of  life.  Developments  and  advances  especially  in  the  areas  of  digital  electronics  and  micro-circuitry  have 
spawned  subsequent  technology  based  improvements  in  transportation,  communications,  entertainment,  automation,  the  armed  forces,  and 
many  other  areas  that  would  not  have  been  possible  otherwise.  This  rapid  “explosion”  of  new  capabilities  and  ways  of  performing  tasks  has 
been  motivated  as  often  as  not  by  the  philosophy  that  if  it  is  possible  to  make  something  better  or  work  faster  or  be  more  cost  effective  or 
operate  over  greater  distances  then  it  must  inherently  be  good  for  the  human  operator.  Taken  further,  these  improvements  typically  are 
envisioned  to  consequently  produce  a  more  efficient  operating  system  where  the  human  operator  is  an  integral  component.  The  formal 
concept  of  human-system  interface  design  has  only  emerged  this  century  as  a  recognized  academic  discipline,  however,  the  practice  of 
developing  ideas  and  concepts  for  systems  containing  human  operators  has  been  in  existence  since  humans  started  experiencing  cognitive 
thought. 

An  example  of  a  human  system  interface  technology  for  communication  and  dissemination  of  written  information  that  has  evolved  over 
centuries  of  trial  and  error  development,  is  the  book.  It  is  no  accident  that  the  form  and  shape  of  the  book  of  today  is  as  it  is.  This  is  because  it 
is  a  shape  and  form  readily  usable  by  human  physiology  whose  optimal  configuration  was  determined  by  centuries  of  effort  and  revision.  This 
slow  evolution  was  mirrored  by  a  rate  of  technical  evolution  in  printing  and  elsewhere  that  allowed  new  advances  to  be  experimented  with  as 
part  of  the  overall  use  requirement  and  need  for  the  existence  of  the  printed  word  and  some  way  to  contain  it.  Today,  however,  technology  is 
advancing  at  such  a  rapid  rate  that  evolutionary  use  requirements  have  no  chance  to  develop  along  side  the  fast  pace  of  technical  progress. 

One  result  of  this  recognition  is  the  establishment  of  disciplines  like  human  factors  engineering  that  have  stated  purposes  and  goals  of 
systematic  determination  of  good  and  bad  human  system  interface  designs.  However,  other  results  of  this  phenomenon  are  systems  that  get 
developed  and  placed  into  public  use  simply  because  new  technology  allowed  them  to  be  made.  This  development  can  proceed  without  a  full 
appreciation  of  how  the  system  might  be  used  and,  perhaps  even  more  significantly,  what  impact  the  use  of  this  new  system  might  have  on  the 
operator  within  it.  (continued  on  next  page) 
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Item  14  (abstract)  continued 

The  U.S.  Army  has  a  term  for  this  type  of  activity.  It  is  called  “stove-piped  development.”  The  implication  of  this 
term  is  that  a  system  gets  developed  in  isolation  where  the  developers  are  only  looking  "up”  and  not  ‘  around. 

They  are  thus  concerned  only  with  how  this  system  may  work  or  be  used  for  its  own  singular  purposes  as  opposed  to 
how  it  might  be  used  in  the  larger  community  of  existing  systems  and  interfaces  or,  even  more  importantly,  in  the 
larger  community  of  other  new  systems  in  concurrent  development.  Some  of  the  impacts  for  the  Army  from  this 
mode  of  system  development  are  communication  systems  that  work  exactly  as  designed  but  are  unable  to  interface 
to  other  communications  systems  in  other  domains  for  battlefield  wide  communications  capabilities.  Having 
communications  systems  that  cannot  communicate  with  each  other  is  a  distinct  problem  in  its  own  right.  However, 
when  developments  in  one  industry  produce  products  that  humans  use  or  attempt  to  use  with  products  from  totally 
separate  developments  or  industries,  the  Army  concept  of  product  development  resulting  from  stove-piped  design 
visions  can  have  significant  implication  on  the  operation  of  each  system  and  the  human  operator  attempting  to  use  it. 

There  are  many  examples  that  would  illustrate  the  above  concept,  however,  one  that  will  be  explored  here  is  the 
Army  effort  to  study,  understand,  and  optimize  its  command  and  control  (C2)  operations.  This  effort  is  at  the  heart 
of  a  change  in  the  operational  paradigm  in  C2  Tactical  Operations  Centers  (TOCs)  that  the  Army  is  now  undergoing. 
For  the  50  years  since  World  War  II  the  nature,  organization,  and  mode  of  the  operation  of  command  organizations 
within  the  Army  has  remained  virtually  unchanged.  Staffs  have  been  organized  on  a  basic  four  section  structure  and 
TOCs  generally  only  operate  in  a  totally  static  mode  with  the  amount  of  time  required  to  move  them  to  keep  up  with 
a  mobile  battlefield  going  up  almost  exponentially  from  lower  to  higher  command  levels.  However,  current 
initiatives  are  changing  all  that  and  while  new  vehicles  and  hardware  systems  address  individual  components  of  the 
command  structures  to  improve  their  operations,  these  initiatives  do  not  necessarily  provide  the  environment  in 
which  the  human  operator  component  of  the  overall  system  can  function  in  a  more  effective  manner. 

This  dissertation  examines  C2  from  a  system  level  viewpoint  using  a  new  paradigm  for  systematically  examining 
the  way  TOCs  operate  and  then  translating  those  observations  into  validated  computer  simulations  using  a 
methodological  framework.  This  paradigm  is  called  Computer  Modeling  Paradigm  And  Simulation  of  Systems 
(COMPASS).  COMPASS  provides  the  ability  to  model  TOC  operations  in  a  way  that  not  only  includes  the 
individuals,  work  groups  and  teams  in  it,  but  also  all  of  the  other  hardware  and  software  systems  and  subsystems  and 
human-system  interfaces  that  comprise  it  as  well  as  the  facilities  and  environmental  conditions  that  surround  it. 

Most  of  the  current  literature  and  research  in  this  area  focuses  on  the  concept  of  C2  itself  and  its  follow-on  activities 
of  command,  control,  communications  (C3),  command,  control,  communications,  and  computers  (C4),  and 
command,  control,  communications,  computers  and  intelligence  (C4I).  This  focus  tends  to  address  the  activities 
involved  with  the  human  processes  within  the  overall  system  such  as  individual  and  team  performance  and  the 
commander’s  decision-making  process.  While  the  literature  acknowledges  the  existence  of  the  command  and 
control  system  (C2S),  little  effort  has  been  expended  to  quantify  and  analyze  C2Ss  from  a  systemic  viewpoint.  A 
C2S  is  defined  as  the  facilities,  equipment,  communications,  procedures,  and  personnel  necessary  to  support  the 
commander  (i.e.,  the  primary  decision  maker  within  the  system)  for  conducting  the  activities  of  planning,  directing, 
and  controlling  the  battlefield  within  the  sector  of  operations  applicable  to  the  system. 

The  research  in  this  dissertation  is  in  two  phases.  The  overall  project  incorporates  sequential  experimentation 
procedures  that  build  on  successive  TOC  observation  events  to  generate  an  evolving  data  store  that  supports  the  two 
phases  of  the  project.  Phase  I  consists  of  the  observation  of  heavy  maneuver  battalion  and  brigade  TOCs  during 
peacetime  exercises.  The  term  “heavy  maneuver”  is  used  to  connotate  main  battle  forces  such  as  armored  and 
mechanized  infantry  units  supported  by  artillery,  air  defense,  close  air,  engineer,  and  other  so  called  combat  support 
elements.  This  type  of  unit  comprises  the  main  battle  forces  on  the  battlefield.  It  is  used  to  refer  to  what  is  called 
the  conventional  force  structure.  These  observations  are  conducted  using  naturalistic  observation  techniques  of  the 
visible  functioning  of  activities  within  the  TOC  and  are  augmented  by  automatic  data  collection  of  such  things  as 
analog  and  digital  message  traffic,  combat  reports  generated  by  the  computer  simulations  supporting  the  wargame 
exercise,  and  video  and  audio  recordings  where  appropriate  and  available.  Visible  activities  within  the  TOC  include 
primarily  the  human  operator  functions  such  as  message  handling  activities,  decision-making  processes  and  timing, 
coordination  activities,  and  span  of  control  over  the  battlefield.  They  also  include  environmental  conditions, 
functional  status  of  computer  and  communications  systems,  and  levels  of  message  traffic  flows.  These  observations 


are  further  augmented  by  observer  estimations  of  such  indicators  as  perceived  level  of  stress,  excitement,  and  level 
of  attention  to  the  mission  of  the  TOC  personnel.  In  other  words,  every  visible  and  available  component  of  the  C2S 
within  the  TOC  is  recorded  for  analysis.  No  a  priori  attempt  is  made  to  evaluate  the  potential  significance  of  each  of 
the  activities  as  their  contribution  may  be  so  subtle  as  to  only  be  ascertainable  through  statistical  analysis.  Each  of 
these  performance  activities  becomes  an  independent  variable  (IV)  within  the  data  that  is  compared  against 
dependent  variables  (DV)  identified  according  to  the  mission  functions  of  the  TOC.  The  DVs  for  the  C2S  are 
performance  measures  that  are  critical  combat  tasks  performed  by  the  system.  Examples  of  critical  combat  tasks  are 
“attacking  to  seize  an  objective”,  “seizure  of  key  terrain”,  and  “river  crossings”.  A  list  of  expected  critical  combat 
tasks  has  been  prepared  from  the  literature  and  subject  matter  expert  (SME)  input.  After  the  exercise  is  over,  the 
success  of  these  critical  tasks  attempted  by  the  C2S  during  the  wargame  are  established  through  evaluator 
assessments,  if  available,  and/or  TOC  staff  self  analysis  and  reporting  as  presented  during  after  action  reviews. 

The  second  part  of  Phase  I  includes  datamining  procedures,  including  neural  networks,  used  in  a  constrained  format 
to  analyze  the  data.  The  term  constrained  means  that  the  identification  of  the  outputs/DV  is  known.  The  process 
was  to  identify  those  IV  that  significantly  contribute  to  the  constrained  DV.  A  neural  network  is  then  constructed 
where  each  IV  forms  an  input  node  and  each  DV  forms  an  output  node.  One  layer  of  hidden  nodes  is  used  to 
complete  the  network.  The  number  of  hidden  nodes  and  layers  is  determined  through  iterative  analysis  of  the 
network.  The  completed  network  is  then  trained  to  replicate  the  output  conditions  through  iterative  epoch 
executions.  The  network  is  then  pruned  to  remove  input  nodes  that  do  not  contribute  significantly  to  the  output 
condition.  Once  the  neural  network  tree  is  pruned  through  iterative  executions  of  the  neural  network,  the  resulting 
branches  are  used  to  develop  algorithmic  descriptors  of  the  system  in  the  form  of  regression  like  expressions. 

For  Phase  II  these  algorithmic  expressions  are  incorporated  into  the  CoHOST  discrete  event  computer  simulation 
model  of  the  C2S.  The  programming  environment  is  the  commercial  programming  language  Micro  Saint  running 
on  a  PC  microcomputer.  An  interrogation  approach  was  developed  to  query  these  algorithms  within  the  computer 
simulation  to  determine  if  they  allow  the  simulation  to  reflect  the  activities  observed  in  the  real  TOC  to  within  an 
acceptable  degree  of  accuracy. 

The  purpose  of  this  dissertation  has  been  to  introduce  the  COMPASS  concept  that  is  a  paradigm  for  developing 
techniques  and  procedures  to  translate  as  much  of  the  performance  of  the  entire  TOC  system  as  possible  to  an 
existing  computer  simulation  that  would  be  suitable  for  analyses  of  future  system  configurations. 

The  approach  consists  of  the  following  steps: 

•  Naturalistic  observation  of  the  real  system  using  ethnographic  techniques. 

•  Data  analysis  using  data  mining  techniques  such  as  neural  networks. 

•  Development  of  mathematical  models  of  TOC  performance  activities. 

•  Integration  of  the  mathematical  into  the  CoHOST  computer  simulation. 

•  Interrogation  of  the  computer  simulation. 

•  Assessment  of  the  level  of  accuracy  of  the  computer  simulation. 

•  Validation  of  the  process  as  a  viable  system  simulation  approach. 
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(Abstract) 

In  today’s  technology  based  society  the  rapid  proliferation  of  new  machines  and  systems 
that  would  have  been  undreamed  of  only  a  few  short  years  ago  has  become  a  way  of  life. 
Developments  and  advances  especially  in  the  areas  of  digital  electronics  and  micro-circuitry 
have  spawned  subsequent  technology  based  improvements  in  transportation,  communications, 
entertainment,  automation,  the  armed  forces,  and  many  other  areas  that  would  not  have  been 
possible  otherwise.  This  rapid  “explosion”  of  new  capabilities  and  ways  of  performing  tasks  has 
been  motivated  as  often  as  not  by  the  philosophy  that  if  it  is  possible  to  make  something  better  or 
work  faster  or  be  more  cost  effective  or  operate  over  greater  distances  then  it  must  inherently  be 
good  for  the  human  operator.  Taken  further,  these  improvements  typically  are  envisioned  to 
consequently  produce  a  more  efficient  operating  system  where  the  human  operator  is  an  integral 
component.  The  formal  concept  of  human-system  interface  design  has  only  emerged  this 
century  as  a  recognized  academic  discipline,  however,  the  practice  of  developing  ideas  and 
concepts  for  systems  containing  human  operators  has  been  in  existence  since  humans  started 
experiencing  cognitive  thought. 

An  example  of  a  human  system  interface  technology  for  communication  and 
dissemination  of  written  information  that  has  evolved  over  centuries  of  trial  and  error 
development,  is  the  book.  It  is  no  accident  that  the  form  and  shape  of  the  book  of  today  is  as  it 
is.  This  is  because  it  is  a  shape  and  form  readily  usable  by  human  physiology  whose  optimal 
configuration  was  determined  by  centuries  of  effort  and  revision.  This  slow  evolution  was 
mirrored  by  a  rate  of  technical  evolution  in  printing  and  elsewhere  that  allowed  new  advances  to 
be  experimented  with  as  part  of  the  overall  use  requirement  and  need  for  the  existence  of  the 
printed  word  and  some  way  to  contain  it.  Today,  however,  technology  is  advancing  at  such  a 
rapid  rate  that  evolutionary  use  requirements  have  no  chance  to  develop  along  side  the  fast  pace 
of  technical  progress.  One  result  of  this  recognition  is  the  establishment  of  disciplines  like 
human  factors  engineering  that  have  stated  purposes  and  goals  of  systematic  determination  of 
good  and  bad  human  system  interface  designs.  However,  other  results  of  this  phenomenon  are 
systems  that  get  developed  and  placed  into  public  use  simply  because  new  technology  allowed 
them  to  be  made.  This  development  can  proceed  without  a  full  appreciation  of  how  the  system 
might  be  used  and,  perhaps  even  more  significantly,  what  impact  the  use  of  this  new  system 
might  have  on  the  operator  within  it. 

The  U.S.  Army  has  a  term  for  this  type  of  activity.  It  is  called  “stove-piped 
development.”  The  implication  of  this  term  is  that  a  system  gets  developed  in  isolation  where 
the  developers  are  only  looking  “up”  and  not  “around.”  They  are  thus  concerned  only  with  how 
this  system  may  work  or  be  used  for  its  own  singular  purposes  as  opposed  to  how  it  might  be 


used  in  the  larger  community  of  existing  systems  and  interfaces  or,  even  more  importantly,  in  the 
larger  community  of  other  new  systems  in  concurrent  development.  Some  of  the  impacts  for  the 
Anny  from  this  mode  of  system  development  are  communication  systems  that  work  exactly  as 
designed  but  are  unable  to  interface  to  other  communications  systems  in  other  domains  for 
battlefield  wide  communications  capabilities.  Having  communications  systems  that  cannot 
communicate  with  each  other  is  a  distinct  problem  in  its  own  right.  However,  when 
developments  in  one  industry  produce  products  that  humans  use  or  attempt  to  use  with  products 
from  totally  separate  developments  or  industries,  the  Army  concept  of  product  development 
resulting  from  stove-piped  design  visions  can  have  significant  implication  on  the  operation  of 
each  system  and  the  human  operator  attempting  to  use  it. 

There  are  many  examples  that  would  illustrate  the  above  concept,  however,  one  that  will 
be  explored  here  is  the  Army  effort  to  study,  understand,  and  optimize  its  command  and  control 
(C2)  operations.  This  effort  is  at  the  heart  of  a  change  in  the  operational  paradigm  in  C2  Tactical 
Operations  Centers  (TOCs)  that  the  Army  is  now  undergoing.  For  the  50  years  since  World  War 
II  the  nature,  organization,  and  mode  of  the  operation  of  command  organizations  within  the 
Anny  has  remained  virtually  unchanged.  Staffs  have  been  organized  on  a  basic  four  section 
structure  and  TOCs  generally  only  operate  in  a  totally  static  mode  with  the  amount  of  time 
required  to  move  them  to  keep  up  with  a  mobile  battlefield  going  up  almost  exponentially  from 
lower  to  higher  command  levels.  However,  current  initiatives  are  changing  all  that  and  while 
new  vehicles  and  hardware  systems  address  individual  components  of  the  command  structures  to 
improve  their  operations,  these  initiatives  do  not  necessarily  provide  the  environment  in  which 
the  human  operator  component  of  the  overall  system  can  function  in  a  more  effective  manner. 

This  dissertation  examines  C2  from  a  system  level  viewpoint  using  a  new  paradigm  for 
systematically  examining  the  way  TOCs  operate  and  then  translating  those  observations  into 
validated  computer  simulations  using  a  methodological  framework.  This  paradigm  is  called 
COmputer  Modeling  Paradigm  And  Simulation  of  Systems  (COMPASS).  COMPASS  provides 
the  ability  to  model  TOC  operations  in  a  way  that  not  only  includes  the  individuals,  work  groups 
and  teams  in  it,  but  also  all  of  the  other  hardware  and  software  systems  and  subsystems  and 
human-system  interfaces  that  comprise  it  as  well  as  the  facilities  and  environmental  conditions 
that  surround  it. 

Most  of  the  current  literature  and  research  in  this  area  focuses  on  the  concept  of  C2 
itself  and  its  follow-on  activities  of  command,  control,  communications  (C3),  command,  control, 
communications,  and  computers  (C4),  and  command,  control,  communications,  computers  and 
intelligence  (C4I).  This  focus  tends  to  address  the  activities  involved  with  the  human  processes 
within  the  overall  system  such  as  individual  and  team  performance  and  the  commander’s 
decision-making  process.  While  the  literature  acknowledges  the  existence  of  the  command  and 
control  system  (C2S),  little  effort  has  been  expended  to  quantify  and  analyze  C2Ss  from  a 
systemic  viewpoint.  A  C2S  is  defined  as  the  facilities,  equipment,  communications,  procedures, 
and  personnel  necessary  to  support  the  commander  (i.e.,  the  primary  decision  maker  within  the 
system)  for  conducting  the  activities  of  planning,  directing,  and  controlling  the  battlefield  within 
the  sector  of  operations  applicable  to  the  system. 

The  research  in  this  dissertation  is  in  two  phases.  The  overall  project  incorporates 
sequential  experimentation  procedures  that  build  on  successive  TOC  observation  events  to 
generate  an  evolving  data  store  that  supports  the  two  phases  of  the  project.  Phase  I  consists  of 
the  observation  of  heavy  maneuver  battalion  and  brigade  TOCs  during  peacetime  exercises. 

The  term  “heavy  maneuver”  is  used  to  connotate  main  battle  forces  such  as  armored  and 
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mechanized  infantry  units  supported  by  artillery,  air  defense,  close  air,  engineer,  and  other  so 
called  combat  support  elements.  This  type  of  unit  comprises  the  main  battle  forces  on  the 
battlefield.  It  is  used  to  refer  to  what  is  called  the  conventional  force  structure.  These 
observations  are  conducted  using  naturalistic  observation  techniques  of  the  visible  functioning  of 
activities  within  the  TOC  and  are  augmented  by  automatic  data  collection  of  such  things  as 
analog  and  digital  message  traffic,  combat  reports  generated  by  the  computer  simulations 
supporting  the  wargame  exercise,  and  video  and  audio  recordings  where  appropriate  and 
available.  Visible  activities  within  the  TOC  include  primarily  the  human  operator  functions  such 
as  message  handling  activities,  decision-making  processes  and  timing,  coordination  activities, 
and  span  of  control  over  the  battlefield.  They  also  include  environmental  conditions,  functional 
status  of  computer  and  communications  systems,  and  levels  of  message  traffic  flows.  These 
observations  are  further  augmented  by  observer  estimations  of  such  indicators  as  perceived  level 
of  stress,  excitement,  and  level  of  attention  to  the  mission  of  the  TOC  personnel.  In  other  words, 
every  visible  and  available  component  of  the  C2S  within  the  TOC  is  recorded  for  analysis.  No  a 
priori  attempt  is  made  to  evaluate  the  potential  significance  of  each  of  the  activities  as  their 
contribution  may  be  so  subtle  as  to  only  be  ascertainable  through  statistical  analysis.  Each  of 
these  perfonnance  activities  becomes  an  independent  variable  (IV)  within  the  data  that  is 
compared  against  dependent  variables  (DV)  identified  according  to  the  mission  functions  of  the 
TOC.  The  DVs  for  the  C2S  are  performance  measures  that  are  critical  combat  tasks  perfonned 
by  the  system.  Examples  of  critical  combat  tasks  are  “attacking  to  seize  an  objective”,  “seizure 
of  key  terrain”,  and  “river  crossings”.  A  list  of  expected  critical  combat  tasks  has  been  prepared 
from  the  literature  and  subject  matter  expert  (SME)  input.  After  the  exercise  is  over,  the  success 
of  these  critical  tasks  attempted  by  the  C2S  during  the  wargame  are  established  through 
evaluator  assessments,  if  available,  and/or  TOC  staff  self  analysis  and  reporting  as  presented 
during  after  action  reviews. 

The  second  part  of  Phase  I  includes  datamining  procedures,  including  neural  networks, 
used  in  a  constrained  format  to  analyze  the  data.  The  tenn  constrained  means  that  the 
identification  of  the  outputs/DV  is  known.  The  process  was  to  identify  those  IV  that 
significantly  contribute  to  the  constrained  DV.  A  neural  network  is  then  constructed  where  each 
IV  forms  an  input  node  and  each  DV  forms  an  output  node.  One  layer  of  hidden  nodes  is  used  to 
complete  the  network.  The  number  of  hidden  nodes  and  layers  is  detennined  through  iterative 
analysis  of  the  network.  The  completed  network  is  then  trained  to  replicate  the  output  conditions 
through  iterative  epoch  executions.  The  network  is  then  pruned  to  remove  input  nodes  that  do 
not  contribute  significantly  to  the  output  condition.  Once  the  neural  network  tree  is  pruned 
through  iterative  executions  of  the  neural  network,  the  resulting  branches  are  used  to  develop 
algorithmic  descriptors  of  the  system  in  the  form  of  regression  like  expressions. 

For  Phase  II  these  algorithmic  expressions  are  incorporated  into  the  CoHOST  discrete 
event  computer  simulation  model  of  the  C2S.  The  programming  environment  is  the  commercial 
programming  language  Micro  Saint™  running  on  a  PC  microcomputer.  An  interrogation 
approach  was  developed  to  query  these  algorithms  within  the  computer  simulation  to  detennine 
if  they  allow  the  simulation  to  reflect  the  activities  observed  in  the  real  TOC  to  within  an 
acceptable  degree  of  accuracy. 

The  purpose  of  this  dissertation  has  been  to  introduce  the  COMPASS  concept  that  is  a 
paradigm  for  developing  techniques  and  procedures  to  translate  as  much  of  the  performance  of 
the  entire  TOC  system  as  possible  to  an  existing  computer  simulation  that  would  be  suitable  for 
analyses  of  future  system  configurations. 
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The  approach  consists  of  the  following  steps: 

•  Naturalistic  observation  of  the  real  system  using  ethnographic  techniques. 

•  Data  analysis  using  datamining  techniques  such  as  neural  networks. 

•  Development  of  mathematical  models  of  TOC  performance  activities. 

•  Integration  of  the  mathematical  into  the  CoHOST  computer  simulation. 

•  Interrogation  of  the  computer  simulation. 

•  Assessment  of  the  level  of  accuracy  of  the  computer  simulation. 

•  Validation  of  the  process  as  a  viable  system  simulation  approach. 
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1.  Introduction. 


Following  the  Persian  Gulf  War,  when  the  U.S.  Anny  determined  that  its  current  armored 
command  and  control  vehicle  was  obsolete,  the  Human  Research  and  Engineering  Directorate  of 
the  U.S.  Anny  Research  Laboratory  began  a  series  of  studies  and  projects  focused  on 
investigating  the  nature  of  military  command  and  control  (C2)  operations.  The  questions  seeking 
answers  were  whether  these  operations  would  be  affected  by  the  new  operational  paradigm 
mandated  with  the  introduction  of  digitized  battlefield  systems  in  the  area  of  command,  control 
and  communications  (C3).  These  initiatives  resulted  in  a  number  of  projects,  products,  and 
studies  aimed  at  addressing  these  issues.  Among  the  modeling  efforts  were  the  IMPRINT 
modeling  tool  for  developing  models  of  individual  and  workstation  soldier  performance 
(Allender,  Kelley,  Archer,  and  Adkins,  1994),  CrewCut  (Dahl,  Laughery,  Hahler,  Lockett,  and 
Thein,  1991;  Hahler,  Dahl,  Laughery,  Lockett,  and  Thein,  1991)  that  provides  an  environment 
for  the  analysis  of  crew  workload,  WinCrew  (Archer  and  Lockett,  1997)  for  modeling  human 
performance  and  workload,  the  JACK  anthropometric  model  (Kozycki,  Faughn,  Leiter,  and 
Lockett,  1997),  and  the  CoHOST  task  and  workload  models  for  battalion  and  brigade  command 
and  control  teams  (Middlebrooks  et  al.,  1999b).  A  series  of  studies  was  also  conducted  to 
investigate  the  nature  of  the  cognitive  aspects  of  battlefield  command  and  control  and  address 
how  the  operational  shift  to  digitized  operations  would  affect  it  (Adelman,  Leedom,  Murphy,  and 
Killam,  1998;  Cook,  Leedom,  Grynovicki,  and  Golden,  2000;  Golden,  Cook,  Grynovicki,  Kysor, 
and  Leedom,  2000;  Leedom,  Adelman,  and  Murphy,  1998;  Leedom  and  Fallesen,  1998;  Leedom, 
Murphy,  and  Adelman,  1998a;  Leedom,  Murphy,  and  Adelman,  1998b;  Murphy,  Adelman, 
Leedom,  Grynovicki,  Golden,  and  Kysor,  1998). 

These  efforts  have  been  in  association  with  many  other  individuals,  groups  and  agencies 
across  the  human  factors  domain  (Bethmann,  Malloy,  and  Hoever,  1989;  Cooper,  Shiflett,  and 
Crotkin,  1984;  DAngelo,  1980;  Levis  and  Athans,  1986;  Maillefert,  1975;  Monguillet  and  Levis, 
1988;  Perdu,  1988;  Runals,  1985;  Sutten  and  Hervey,  1986;  Walker,  Reimer,  Brown,  and 
Kloecker,  1984;  Wildenberg,  1987;  Wohl,  Entin,  and  Eterno,  1983).  These  have  all  met  with 
varying  degrees  of  success  and  have  typically  addressed  the  issue  from  a  specific  point  of 
reference.  Examples  of  these  points  of  reference  include  “command  and  control  operations” 
(Ainslie,  Leibrecht,  and  Atwood,  1991;  Bolte,  Black,  and  Mendel,  1991;  Crumley  and  Shennan, 
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1990;  Fallesen,  Lussier,  and  Michel,  1992a;  McCann,  1990;  Olsen,  1991;  Swanson  and  Gibson, 
1990;  Zubal  and  Steinberg,  1989),  “team  performance”  (Bowers,  Jentsch,  and  Morgan,  2001; 
Campion,  Brander,  and  Koritsas,  1998;  Kay  and  Dolgin,  1998;  Mathieu,  Heffner,  Goodwin, 
Salas,  and  Cannon-Bowers,  2000;  Militello,  Kyne,  Klein,  Getchell,  and  Thordsen,  1999;  Peters, 
1997;  Sebok,  2000),  “task  analysis’  to  include  cognitive  task  analysis  (Ainsworth,  2001;  Kieras 
and  Meyer,  2000;  Klein,  2000b;  Luczak  and  Stahl,  2001;  McNeese  and  Rentsch,  2001;  Prietula, 
Feltovich,  and  Marchak,  2000;  Schaafstal,  Schraagen,  and  van  Berio,  2000;  Schraagen, 

Chipman,  and  Shalin,  2000a;  Schraagen,  Chipman,  and  Shalin,  2000b;  Schraagen,  Chipman,  and 
Shute,  2000;  Seamster,  Redding,  and  Kaempf,  2000;  Vicente,  2000),  and  cognitive  “task  and 
workload”  assessment  (Adams,  Tenney,  and  Pew,  1991;  Hamilton,  Bierbaum,  and  Fulford,  1991; 
Laughery,  1989a;  Middlebrooks,  2001;  Middlebrooks  et  al.,  1999b;  Wierwille,  Rahimi,  and 
Casali,  1986;  Xie,  1997). 

This  work  has  predominately  focused  on  the  human  component  in  the  human  systems 
interface  with  results  that  typically  address  issues  that  relate  to  the  system  operator(s).  The  goal 
of  this  research  is  to  develop  methodologies,  modeling  tools,  and  initial  products  that  address  the 
total  system  that  encompasses  a  battalion  or  brigade  command  and  control  “operation.”  The 
tenn  “operation”  is  used  here  to  imply  the  entire  C2  system.  This  includes  the  human  team 
members,  digital  command  and  control  systems,  and  external  influences  that  affect  the 
performance  of  this  system.  Because  of  the  potentially  huge  numbers  of  variables  and  resultant 
data,  this  project  started  with  the  realization  that  the  extent  of  the  data  may  not  be  known  until 
after  it  had  been  collected.  At  this  point  the  final  analysis  plan  could  only  then  be  finalized.  It 
was  presumed  that  traditional  predictive  techniques  such  as  full  and  partial  factorial 
experimentation  and  linear,  multiple,  and  polynomial  regression  will  be  inefficient  to  attempt.  In 
this  research,  these  techniques  are  supplanted  by  datamining  approaches  that  attempt  to  identify 
significant  relationships  in  very  large  datasets. 

This  research  had  the  potential  to  fall  within  one  of  two  broad  categories.  A  content- 
oriented  research  effort,  in  the  context  of  tactical  operations  center  (TOC)  operations,  would 
focus  on  the  issue  of  how  to  design  better  TOCs  and  would  have  the  primary  goal  of  identifying 
key  significant  parameters  that  directly  affect  how  a  TOC  system  perfonns.  The  associated 
processes  and  procedures  that  dictate  how  these  independent  variable  (IV)  parameters  would  be 
used  to  evaluate  existing  TOC  designs  would  be  a  byproduct  of  an  effort  focused  on  the 
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development  of  a  better  understanding  of  how  a  TOC  operates  and  what  factors  significantly 
affect  that  operation.  Of  follow-on  importance  would  be  the  ability  to  predict  the  operation  of 
future  TOC  designs  and/or  impacts  of  new  TOC  subsystem  employments. 

Alternatively,  a  methodology-based  research  effort  would  focus  on  the  development  of  a 
framework  to  describe  the  processes  and  procedural  tools  that  can  be  used  to  evaluate  the 
performance  of  human-based  systems  such  as  military  TOCs.  These  methodologies  would  have 
potential  application  to  other  similar  systems  where  time  pressure  and  high  stress  levels  play 
critical  roles  in  the  performance  of  the  overall  system  and  its  human  component  subsystem.  This 
methodological  framework  would  be  developed  using  military  TOCs  as  the  exploratory  medium 
and  the  process  of  developing  the  procedures  would  provide  insight  into  critical  elements  of 
TOC  operations.  However,  the  primary  goal  of  the  research  would  be  to  develop  techniques  and 
procedures  that  would  be  applicable  to  a  wide  range  of  team-based,  performance-oriented  work 
groups  and  systems. 

The  distinction  between  using  content  versus  methodological  approaches  for  this  research 
is  subtle,  but  significant.  For  this  research  both  approaches  have  similar  end  states  as  overall 
project  objectives,  however,  the  priorities  to  be  followed  during  the  course  of  the  work  are 
significantly  different.  Since  this  work  is  but  the  beginning  of  a  long-term  goal  of  establishing  a 
new  research  program  at  ARL  to  investigate  and  improve  TOC  perfonnance,  the  overall  project 
is  envisioned  to  have  a  content-oriented  objective.  However,  since  the  purpose  of  this 
dissertation  is  to  develop  a  framework  of  methodologies  and  procedures  that  will  be  used  for  the 
on-going  effort,  the  research  focus  is  methodological  in  nature. 

1.1.  Research  Goals  and  Objectives. 

The  purpose  of  this  dissertation  is  to  take  a  systemic  look  at  an  activity  the  U.S.  Army 
has  been  actively  trying  to  understand  for  at  least  the  last  50  years.  Anny  command  and  control 
work  teams  have  many  similarities  to  counterparts  in  the  civilian  work  sector  where  time 
pressure  and  high  stress  play  critical  roles.  Counterpart  examples  from  the  civilian  sector 
include  hospital  emergency  room  teams,  nuclear  power  plant  operator  teams,  and  ship  handling 
teams.  What  sets  Army,  and  other  military,  teams  apart  from  these  counterparts  are  the  dire 
consequences  that  could  arise  from  ineffective  performance.  Instantaneous  life  and  death, 
potentially  across  the  entire  battlefield  that  could  encompass  the  team  itself,  are  possible  results 
of  how  well  the  team  performs  in  its  work  domain. 
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While  much  of  the  literature  focuses  on  how  well  the  human  component  of  the  overall 
system  performs  (i.e.,  the  team),  this  research  has  a  goal  of  examining  the  performance  of  the 
total  command  and  control  system.  The  focus  is  on  Army  battalion  and  brigade  command  and 
control  centers  as  these  two  systems  are  very  similar  in  organization  and  scope.  In  the  U.S. 
Anny  the  battalion,  which  consists  of  approximately  800  soldiers,  is  the  lowest  level  where  the 
formal  command  and  control  system  is  in  existence.  The  brigade  system,  at  the  next  higher 
level,  directs  the  activities  of  three  battalions.  The  complexity  of  the  battlespace  management 
tasks  can  be  illustrated  by  looking  at  a  tactical  operations  map  used  by  a  TOC  during  combat 
operations.  Figure  1  is  an  example  of  this  type  of  map  where  tactical  control  measure  lines  and 
symbols  representing  units  on  the  battlefield  is  overlayed  on  top  of  a  topographic  map.  The 
legend  indicates  the  symbol  color  for  different  friendly  units  on  the  map  while  enemy  units  are 
shown  in  red.  A  thermometer  like  color  filling  of  the  unit  symbols  is  used  to  represent 
approximate  the  strength  of  the  unit.  The  image  in  this  figure  was  produced  from  electronic 
imagery  generated  from  a  computer  wargame  simulation.  While  some  units  still  use  paper  maps 
with  manually  posted  unit  symbol  stickers  during  training  exercises,  there  are  digital  battlefield 
systems  in  testing  and  initial  trial  use  that  provide  this  type  of  electronic  imagery  to  the  TOC 
from  actual  battlefield  intelligence  data. 
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Figure  1  -  Example  of  a  Simulated  Electronic  Tactical  Operations  Map 


1.1.1.  Research  Objective. 

As  the  Army  is  struggling  with  the  issues  surrounding  the  change  in  the  operational 
paradigm  for  how  a  TOC  is  to  perfonn  in  a  digitized  and  mobile  environment,  techniques  and 
procedures  need  to  be  developed  that  will  allow  the  identification  of  key  parameters  that  should 
be  included  in  TOC  design.  It  is  the  intent  of  this  dissertation  to  develop  a  framework  of 
techniques  and  procedures  that  are  based  upon  empirical  methods  but  are  usable  in  real  world, 
uncontrolled  environments  which  can  only  be  approached  using  techniques  such  as  naturalistic 
and  ethnographic  observation  methods.  Examples  of  some  real  world  environments  in  addition 
to  the  military  TOCs  include  hospital  emergency  rooms,  shipboard  bridges,  and  nuclear  power 
plant  control  rooms.  These  are  all  examples  of  high  stress,  time  critical  work  systems  where  the 
system  performs  in  accordance  to  its  input  stressors  in  order  to  meet  established  perfonnance 
guidelines.  As  such,  they  are  totally  intolerant  to  the  application  of  any  attempt  at  external 
control  such  as  would  be  represented  by  experimental  design  treatment  conditions.  In  order  to 
achieve  the  desired  fidelity  in  the  analytical  conclusions,  the  data  must  be  rigorous  enough  to 
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support  empirical  inquiry  but  must  be  recorded  in  an  unconstrained,  non-experimental 
environment  where  the  process  of  gathering  the  data  must  not  change  the  performance  of  the 
system  being  observed. 

1.1.2.  Development  of  Procedures  that  will  Allow  the  Screening  and  Identification  of 

Significant  IVs  for  TOC  Simulations. 

The  operation  of  military  C2  is  a  complicated  arena  where  many  different  factors  can 
have  an  impact  on  the  outcome  of  the  performance  of  the  system.  While  previous  studies  have 
identified  63  IV,  the  potential  exists  that  this  list  could  grow  to  100  or  more  as  more  data  are 
identified  and  gathered.  In  addition,  while  previous  studies  have  considered  6  IV  from  a  list  of 
22  possible  performance  measures,  future  studies  could  consider  much  larger  performance 
specifications. 

The  consideration  of  data  sets  of  this  size  requires  techniques  and  procedures  that 
transcend  normal  statistical  inquiry.  First,  because  different  IV  will  be  observed  from  different 
observation  events,  the  use  of  sequential  experimentation  methods  (Han,  Williges,  and  Williges, 
1997;  Williges,  Williges,  and  Han,  1992;  Williges,  Williges,  and  Han,  1993)  is  warranted  as  a 
systematic  process  to  gather  and  organize  the  data.  Once  the  data  is  gathered,  however,  some 
process  is  needed  to  identify  meaningful  relationships  in  the  dataset  and,  further,  to  derive 
meaningful  conclusions  from  that  data.  There  is  also  the  need  to  be  able  to  identify  the  key 
parameters  or  IV  that  significantly  describe  the  operation  of  the  TOC  so  that  these  parameters 
can  then  be  used  in  predictive  computer  simulations  of  the  TOC.  There  are  existing  methods  that 
can  be  applied  to  assist  in  this  analysis  effort,  however,  a  framework  that  combines  these 
methods  into  a  new  analytical  paradigm  is  needed.  Some  of  the  techniques  that  are  candidates  to 
be  incorporated  into  this  paradigm  include  neural  network  simulation,  regression  analysis,  and 
cluster  analysis.  Subject  matter  expert  opinion,  while  not  an  analytical  procedure,  can  also  play  a 
significant  role  in  the  detennination  of  critical  system  components.  These  methods  are 
summarized  in  the  following  paragraphs: 

•  Neural  Network  Simulation.  Neural  networks  (Kwahk,  2002;  Ntuen  and  Li,  2000; 
Reilly  and  Cooper,  1995)  have  the  unique  ability  to  investigate  very  large 
databases  that  contain  many  different  factors.  The  drawback  to  their  use  is  that 
the  final  results  can  be  dependent  on  the  technical  ability  and  subjective  opinion 
of  the  analyst  as  to  when  the  network  has  achieved  a  state  of  being  fully  trained. 
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Further  subjective  opinion  is  required  to  identify  the  level  of  pruning  that  is 
appropriate  for  the  system  being  evaluated.  As  a  result,  the  use  of  NNS  can  be 
considered  an  inexact  science.  However,  this  is  probably  not  more  so  than  many 
other  multivariate  techniques  where  the  problem  space  must  be  probed  with 
attempts  to  identify  significant  relationships  between  factors  in  the  dataset. 

•  Regression  Analysis.  Before  the  advent  of  computer  based  statistical  packages  a 
regression  analysis  suffered  the  same  problem  of  full  factorial  ANOVA 
experimentation.  For  small  evaluations  of  up  to  3  or  4  factors,  regression 
analyses  could  be  developed  using  manual  means.  Beyond  this  they  became  very 
large  and  inefficient  to  attempt.  With  computer  based  statistical  analysis  many  of 
these  limitations  have  been  eliminated.  However,  early  computer  based  packages 
that  ran  on  mainframe  computers  in  a  batch  oriented  mode  were  difficult  and 
cumbersome  to  use  and  the  turnaround  from  submission  of  a  statistical  analysis 
run  to  the  receipt  of  results  could  be  days  or  even  weeks.  The  advent  of  powerful 
desktop  PC  based  computers  has  only  recently  afforded  a  true  interactive  interface 
with  the  ability  to  handle  large  regression  situations  that  could  not  have  been 
considered  only  a  short  time  ago.  If  this  approach  can  handle  the  large  number  of 
IV  and  dependent  variables  (DV)  that  are  expected  then  the  advantage  would  be  a 
deterministic  numerical  solution  that  provides  a  very  close  mathematical 
description  of  the  problem  space. 

•  Cluster  Analysis.  Cluster  Analysis  (CA)  is  the  generic  name  for  mathematical 
models  that  can  be  used  to  find  out  which  objects  in  a  set  are  similar  (Romesburg, 
1984).  It  also  allows  the  ability  to  find  out  which  objects  in  a  set  are  dissimilar. 
The  most  widely  used  fonn  of  CA  is  hierarchical  CA  (HCA).  The  steps  in  HCA 
are: 

1 .  Collect  a  data  matrix  where  the  columns  consist  of  the  objects  to  be 
cluster  analyzed  and  the  rows  are  attributes  that  describe  the  objects. 

2.  Standardize  the  data  matrix  (optional). 

3.  Compute  the  values  of  a  resemblance  coefficient  as  a  measure  of  the 
similarities  among  all  the  pairs  of  the  data  objects  (from  the 
standardized  data  matrix). 
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4.  Process  the  values  of  the  resemblance  coefficient  using  a  clustering 
method  which  results  in  a  tree  diagram  or  dendrogram  that  graphically 
shows  the  relationship  between  the  similarities  among  all  the  pairs  of 
objects. 

•  Subject  Matter  Expert  Opinion.  This  source  of  infonnation  can  provide  a  rich 
store  of  data  that  is  pertinent  and  current  to  the  issues  being  investigated.  Subject 
Matter  Experts  (SME)  have  a  high  degree  of  experience  and  knowledge  that  can 
be  directly  applied  to  a  better  understanding  of  the  system.  These  individuals  can, 
however,  often  be  hard  to  find  and  they  may  not  always  be  willing  to  participate 
in  the  study.  The  opinions  from  SME  can  also  be  highly  subjective  in  nature  and 
many  different  individuals  may  need  to  be  approached  before  a  consistent  opinion 
can  be  realized. 

This  approach  makes  use  of  existing  techniques  and  others  that  may  be  identified  during 
the  course  of  the  study  to  develop  a  systematic  framework  that  can  be  used  to  select  key 
parameters  that  significantly  describe  the  problem  space.  If  a  few  number  of  IV  can  become 
known  that  address  a  large  percentage  of  the  response  of  the  system  then  these  IV  are  the  logical 
candidates  to  be  used  to  cause  a  simulation  of  that  system  to  respond  to  its  stimuli  to  within 
acceptable  limits  of  precision.  Assuming  success  in  this  effort,  then,  these  few  IV  can  be  used 
for  multiple  purposes.  First,  they  are  the  direct  indicators  of  those  factors  that  system  designers 
should  account  for  in  the  design  of  the  system.  Second,  they  provide  the  parameters  that  must  be 
included  in  a  computer  simulation  of  that  system  in  order  to  allow  that  simulation  to  be  used  for 
predictive  purposes  related  to  new  system  designs  and  proposed  modifications.  It  is  the  merging 
and  tailoring  of  all  the  original  techniques  and  methods  into  a  consistent  methodological 
procedure  or  paradigm  that  will  be  the  contribution  of  this  research  topic.  This  new  paradigm 
would  then  be  demonstrated  using  the  case  of  the  military  TOC  to  illustrate  how  key  parameters 
can  be  identified  that  describe  the  significant  performance  of  the  TOC. 

1.1.3.  Development  of  a  new  Framework  for  Evaluating  the  Performance  of  a  C2S. 

The  development  of  an  integrated  framework  to  screen  and  identify  significant  IV s  for 
TOC  simulations  is  the  focus  of  this  research  because  it  is  considered  to  be  a  logical  starting 
point  for  what  is  envisioned  to  be  a  continuing  series  of  studies  that  will  comprise  a  total  project 
program  that  will  continue  past  this  dissertation.  This  integrated  framework  is  a  new  paradigm 


for  a  direct  path  from  the  observation  of  unconstrained,  real  system  performance  and  the  ability 
to  accurately  model  it  in  computer  simulation.  This  new  paradigm  is  called  COmputer  Modeling 
Paradigm  And  Simulation  of  Systems  (COMPASS). 

The  COMPASS  integrated  framework,  as  shown  in  Figure  2,  can  be  summarized  as  a 
systematic  approach  that  provides  an  ability  to  look  at  a  highly  complex  system  that  contains  so 
many  operational  parameters  as  to  make  it  virtually  unapproachable  using  conventional 
experimental  techniques  such  as  factorial  experimentation  and  regression  analysis.  This 
framework  begins  with  system  level  observations  of  perfonnance  of  the  real  system  using 
naturalistic  observations  with  an  ethnographic  approach.  It  derives  pertinent  information 
concerning  key  parameters  that  define  the  perfonnance  of  that  system  to  an  identified  level  of 
confidence  using  exploratory  approaches  such  as  NNS.  These  parameters  are  then  utilized  in 
discrete  event  computer  simulations  of  the  system  to  provide  the  ability  to  model  overall  system 
performance  and  establish  a  platform  for  evaluating  changes  and  modifications  that  may  be 
projected  for  different  portions  of  subsystems  within  the  total  system.  The  validation  of  the 
resulting  simulation  is  then  the  comparison  of  its  response  back  to  the  original  observations  of 
that  system. 

COMPASS  consists  of  the  specific  observation  process,  the  NNS  exploratory  process 
generation  of  descriptive  and  mathematical  models,  application  of  these  models  to  a  discrete 
event  computer  simulation,  and  the  validation  of  the  results  of  that  simulation  back  to  the 
original  environmental  conditions.  The  successful  completion  of  the  COMPASS  procedures  will 
produce  a  resultant  computer  simulation  that  not  only  is  capable  of  performing  evaluative  studies 
on  the  original  system,  but  will  be  flexible  to  updates  from  iterative  uses  of  the  methodology  to 
keep  pace  with  evolving  changes  in  the  system  being  evaluated. 

The  contribution  of  this  research  will  be  the  development  of  the  COMPASS  paradigm  for 
assessing  the  performance  of  complex  systems  that,  in  reality,  are  systems  of  systems  that  are 
reacting  to  the  influence  of  hundreds  of  operational  parameters  and  whose  measure  of 
performance  can  be  spread  across  dozens  of  response  characteristics.  While  COMPASS  will 
make  use  of  many  techniques  and  procedures  already  in  existence,  it  is  the  integration  of  these 
methods  combined  with  novel  use  of  exploratory  capabilities  contained  in  multivariate 
datamining  techniques  such  as  NNS  that  will  make  it  a  potentially  new  and  powerful  approach 
for  investigating  systems  that  have  previously  eluded  systematic  evaluation. 
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Figure  2  -  COMPASS  Concept 


1 .2.  The  Study  of  Battle  Command  in  the  U.S.  Army. 

An  understanding  of  some  of  the  concepts  and  techniques  associated  with  U.S.  Army 
doctrine  for  the  control  of  land  forces  during  periods  of  combat  is  essential  to  attempts  to 
quantify,  evaluate,  and  improve  battle  command  procedures.  Several  topical  areas  are  reviewed 
in  preparation  for  the  presentation  of  the  COMPASS  paradigm. 

1.2.1.  The  Problem  of  C2  Analysis  and  Research. 

Military  C2  is  described  as  the  process  by  which  commanders  organize  and  employ 
forces  to  achieve  military  objectives  (Mason  and  Moffat,  2000).  Simulation  and  modeling  of 
military  C2  is  recognized  as  one  of  the  most  challenging  areas  in  defense  analysis.  Although 
NATO,  for  example,  has  been  investigating  military  C2  for  many  years  with  great  expenditure, 
little  progress  has  been  made.  The  vagaries  of  human  decision  making  makes  C2  extremely 


10 


difficult  to  analyze.  In  fact,  NATO’s  Research  Study  Group  on  Modeling  of  Command  and 
Control  has  concluded  that  “no  single  measure  exists  that  satisfactorily  allows  the  assessment  of 
either  the  overall  effectiveness  of  C2  or  the  performance  of  C2  systems”  (Jablunovsky,  Dorman, 
and  Yoworsky,  2000). 

The  modem  problem  of  C2  is  distinguished  from  its  predecessors  by  the  rate  at  which 
information  must  be  handled  and  the  resultant  decisions  that  must  be  made  utilizing  that 
information  (Adelson,  1961).  Battlefield  C2  is  described  as  a  constant  balancing  act  where  the 
need  for  detailed  planning  must  be  constantly  weighed  against  the  requirement  for  quick  and 
decisive  action  (Fallesen  et  ah,  1992a).  The  characteristics  of  modern  C2  environments  include 
time  pressure,  high  risk,  and  ambiguous  or  missing  information  (Kaempf,  Klein,  Thordsen,  and 
Wolf,  1996)  while  the  goal  of  C2  organizations  is  to  exercise  decisive  control  of  that  hostile 
environment  (Bent,  1983). 

Timing  has  long  been  recognized  as  being  of  critical  importance  in  combat  with  the 
associated  implications  for  C2  (Cothier  and  Levis,  1986).  Some  definitions  related  to  time  and 
C2  are: 


•  System  response  time  is  the  time  delay  between  the  moment  when  the  C3  system 
receives  a  stimulus  and  the  moment  it  can  deliver  a  response. 

•  Tempo  of  operations  is  the  number  of  actions  per  unit  of  time,  which  the  system  is 
executing  and  is  indicative  of  how  complex  the  environment  is  that  the  system 
can  handle. 

•  Scenario  describes  those  actions  that  are  actually  taking  place. 

•  Timeliness  is  the  systems  ability  to  respond  within  an  allotted  time. 

•  Allotted  time  is  the  time  interval  over  which  the  forces  including  the  C3  system 
can  affect  the  environment. 

•  The  system  consists  of  components,  their  interconnections,  and  a  set  of  operating 
procedures. 

•  Boundaries  are  what  defines  what  is  included  within  the  system  whose 
effectiveness  is  to  be  assessed. 

•  The  environment  consists  of  our  own  forces  and  the  adversary’s  forces  upon 
which  our  forces  can  act  and  which  can  act  upon  ours. 

•  Parameters  are  the  independent  quantities  used  to  specify  the  system  and  the 
mission  requirements. 
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•  Measures  of  performance  (MOP)  are  quantities  that  describe  system  properties  or 
mission  requirements.  Example  MOPs  for  a  C2  system  may  include  reliability, 
survivability,  cost,  and  probability  of  kill. 

•  Measures  of  effectiveness  are  quantities  that  result  from  the  comparison  of  the 
system  MOPs  to  the  mission  requirements.  MOEs  reflect  the  extent  to  which  the 
systems  meets  the  requirements. 

One  critical  aspect  of  time  in  the  performance  of  C2  organizations  is  that  it  is  likely  that  requests 
for  services  will  exceed  the  capability  of  even  the  best  C2  system  (Witus  and  Blum,  1992).  C2  is 
performed  at  all  levels  of  military  command,  however,  tactical  C2  is  described  as  a  cyclical 
decision  making  and  communication  process  perfonned  under  rigorous  time  constraints  (Corker, 
Cramer,  Henry,  and  Smoot,  1990).  One  view  of  tactical  C2  describes  it  as  having  4  steps: 

1 .  Situation  assessment. 

2.  Development  of  a  course  of  action. 

3.  Execution  of  that  course  of  action. 

4.  Feedback  of  the  results  of  that  execution. 


Reflecting  these  steps,  a  realistic  model  of  the  C2  environment  should  include  decision-making, 
taking  account  of  the  chain  of  command,  course  of  action  selection,  and  the  communication 
process.  The  bottom  line  is  that  on  the  rapid  paced  battlefield  of  today,  there  is  rarely  enough 
time  or  resources  to  follow  a  systematic  approach  to  achieve  an  optimum  solution  (Fallesen, 
Lussier,  and  Michel,  1992b). 

C2  organizations  are  unique  from  other  forces  on  the  battlefield  as  their  primary  role  is 
not  to  engage  the  enemy  directly,  but  to  direct  the  activities  of  those  forces  who  are  engaging  the 
enemy.  Some  of  the  properties  that  tend  to  distinguish  C2  organizations  are  (Sutherland,  1990): 

•  The  units  of  the  organization  are  both  geographically  dispersed  and  functionally 
specialized. 

•  Each  of  the  individual  units  constituting  a  command  and  control  organization  will 
have  critical  missions  they  are  expected  to  perform. 

•  The  organization  will  have  to  contend  with  one  or  more  ingenious,  aggressive 
adversaries  with  few  if  any  effective  constraints  on  the  nature  of  the  competitive 
initiatives  they  might  author. 

•  The  key  working  challenge  for  the  organization  in  aggregate  and  each  of  the  units 
in  particular  is  to  be  ready  to  deal  with  any  threatening  initiatives  that  might  be 
mounted  by  their  effective  peers  in  the  adversary  organizations  (enemy). 
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Military  forces  throughout  the  world  have  developed  an  awareness  of  the  systems 
approach  to  C2  (Harper,  1974),  where  a  system  is  thought  of  as  a  collection  of  elements 
organized  to  perform  a  set  of  functions,  during  a  specifiable  era,  in  an  environment  known  only 
uncertainly  (Adelson,  1961). 

1.2.2.  Data  Overload  and  Its  Effect  on  C2. 

One  of  the  elements  of  interest  in  the  simulation  and  modeling  of  any  system  where  the 
human  is  a  component  is  cognitive  saturation.  In  other  words,  at  what  point  is  the  human 
operator  perfonning  at  the  full  potential  level  and  is  not  capable  of  performing  additional  tasks  or 
work  without  either  suffering  a  degradation  of  total  performance  or  by  shedding  some  existing 
tasks.  In  command  and  control  systems  the  summation  effects  of  total  task  performance,  which 
may  or  may  not  be  combined  with  moderating  effects  of  such  elements  as  fatigue,  noise,  and 
vibration,  can  induce  what  has  been  called  a  “cognitive  causality”  where  the  individual  is  no 
longer  capable  of  continued  work  or  task  perfonnance  (Middlebrooks,  2001).  Stated  in  another 
way,  data  overload  is  a  condition  where  an  operator,  supported  by  the  physical  components  of 
the  system  and  other  operators,  finds  it  extremely  challenging  to  focus  in  on,  assemble,  and 
synthesize  the  significant  portion  of  the  data  pertaining  to  the  task  or  work  being  executed  in  a 
coherent  manner.  This  generally  is  where  the  task  being  performed  is  a  small  portion  of  the 
requirements  of  the  overall  system  (Patterson,  Roth,  and  Woods,  2002).  Some  of  the 
characteristics  of  data  overload  are  (Woods,  Patterson,  and  Roth,  2002): 

•  A  clutter  problem  where  there  is  too  much  data  on  the  screen. 

•  A  workload  bottleneck  where  there  is  too  much  to  do  in  the  time  available. 

•  A  problem  in  finding  the  significance  of  data  when  it  is  not  known  a  priori  what 
data  from  a  large  data  field  will  be  informative. 

Adding  more  technology,  by  itself,  is  usually  not  enough  to  solve  generic  and  difficult  problems 
like  data  overload  that  are  problems  that  exist  at  the  intersections  of  cognition,  collaboration,  and 
technology.  Data  overload  is  a  condition  where  an  operator,  supported  by  artifacts  and  other 
operators,  finds  it  extremely  challenging  to  focus  in  on,  assemble,  and  synthesize  significant 
portions  of  a  set  of  data  into  a  valid  situation  assessment  where  the  set  of  data  being  considered 
is  part  of  a  vast  data  field  (Patterson  et  al.,  2002). 
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One  reason  that  data  overload  causes  problems  in  C2  organizations  is  the  relationship  of 
data  to  interests  and  expectations  (Woods  et  al.,  2002).  While  new  technology  often  has  the 
stated  purpose  of  easing  this  condition,  it  often  exacerbates  it  by  impressing  upon  the  user  greater 
amounts  of  communications  whose  context  must  be  analyzed  before  it  becomes  useful.  It  is 
noted  that  the  human  ability  to  assimilate  information  from  artificial  fields  of  data  has  expanded 
much  more  slowly,  if  at  all,  than  the  abilities  of  new  technology  to  provide  that  data.  The 
paradox  of  this  situation  is  that  while  more  and  more  data  is  available,  in  principle,  to  those  who 
need  it  from  increased  technology,  the  human  ability  to  interpret  and  understand  what  is 
available  has  not  increased.  Add  to  this  the  typical  battlefield  stressors  of  fatigue  and  stress  and 
the  results  from  cognitive  overload  often  cause  errors  as  human  operators  perform  task 
operations  (Braun  et  al.,  1999). 

1.2.3.  Workload  and  Its  Relationship  to  C2. 

Although  it  is  a  subject  of  intense  investigation,  analysis,  and  debate,  the  simple  fact  of 
the  matter  is  that  nobody  seems  to  know  what  workload  is  (Xie,  1997).  While  it  can 
conceptually  be  defined  as  the  amount  of  mental  work  or  effort  that  an  individual  makes  to 
perform  tasks,  and  can  be  viewed  as  a  consequence  of  human  mental  effort,  one  aspect  seems  to 
be  recognized.  This  conclusion  is  that  mental  workload  is  significantly  affected  by  time 
pressure.  Hence,  the  effects  of  timing  in  C2,  as  just  discussed,  becomes  a  critical  component  of 
mental  workload. 

While  workload  may  not  be  fully  understood,  there  are  significant  and  continuous  efforts 
both  in  the  human  factors  and  psychology  communities  to  establish  its  meaning  and  effects. 
Computer  simulation  plays  a  key  role  in  this  effort  and  in  recognition  of  the  fact  that  new  system 
configurations  may  have  the  unintended  effect  of  increasing  operator  workload  and,  therefore, 
contribute  to  accidents  and  errors,  significant  effort  has  been  devoted  to  the  development  of 
workload  prediction  models  (Dahl  et  al.,  1991).  These  efforts,  although  to  date  are  not 
conclusive  on  exactly  what  is  mental  workload,  have  nevertheless,  developed  significant  levels 
of  understanding  on  various  aspects  of  workload  and  some  insight  on  what  it  is  not.  For 
example,  one  author  notes  that  it  is  widely  accepted  that  the  relationship  between  workload  and 
individual  performance  is  characterized  by  a  curvilinear  function  where  perfonnance  degrades  at 
low  and  high  levels  of  workload  (Bowers,  Braun,  and  Morgan,  1997).  There  are  separate  but 
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parallel  efforts  that  look  at  individual  versus  teamwork  and  workload.  For  example,  one  study 
found  that  the  effect  of  communication  on  team  coordination  did  facilitate  team  performance 
(Williges,  Johnston,  and  Briggs,  1966). 

1 .2.4.  Critical  Incidents  on  the  Battlefield. 

As  is  the  case  with  any  team-oriented  system  that  has  human-in-the-loop  types  of 
activities,  the  identification  of  critical  tasks  that  must  be  performed  can  form  the  basis  for 
studying  the  performance  of  the  team  and  system  itself.  Table  1  shows  a  list  of  these  critical 
tasks.  For  battlefield  command  and  control  these  critical  tasks  are  the  critical  incidents  that 
occur  in  battlefield  operations  and  are  identified  as  those  major  combat  oriented  tasks  on  the 
battlefield  that  must  be  directed  and  coordinated  by  the  C2  system.  A  partial  list  includes  those 
things  that  the  decision  maker  or  commander  may  wish  to  analyze  (Fallesen  et  ah,  1992a): 
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Table  1  -  Battlefield  Critical  Tasks 


_ Combat  Tasks _ 

1 .  Departure  from  the  assembly  area. 

2.  Passage  of  lines. 

3.  Movement  to  the  line  of  departure. 

4.  Breach  of  main  obstacle  belt. 

5.  Penetration  of  defensive  positions. 

6.  Reaction  to  counterattack  forces. 

7.  River  crossing. 

8.  Seizure  of  key  terrain. 

9.  Seizure  of  objective. 

10.  Destruction,  capture,  or  bypass  of  enemy  force. 

11.  Fixing  enemy  in  position. 

12.  Synchronization  with  supporting  forces. 

13.  Use  of  reserves. 

14.  Deep  operations. 

15.  Destruction  of  first  echelon  forces. 

16.  Destruction  of  follow-on  forces. 

17.  Commitment  of  counterattack  forces. 

18.  Deception  activities. 

19.  Rear  operations. 


The  Army  categorizes  these  battlefield  functions  into  four  types  of  operations  that  include 
offensive,  defensive,  stability,  and  support.  These  are  outlined  in  Table  2. 

Table  2  -  Tactical  Battlefield  Functions 
(Army,  2001c) 


Category 

Forms 

Function 

Offensive 

Maneuver 

Envelopment 

Operations 

Turning  Movement 

Infiltration 

Penetration 

Frontal  Attack 

Types 

Movement  To  Contact 

Attack 
-Flasty 
-Deliberate 
-Special  Purpose 
+  Spoiling 
+  Counterattack 
+  Raid 
+  Ambush 
+  Feint 

+  Demonstration 

Exploitation 

Pursuit 
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Category 

Forms 

Function 

Defensive 

Operations 

Types 

Mobile  Defense 

Area  Defense 

Retrograde  Defense 
-Withdrawal 
-Delay 
-Retirement 

Stability 

Operations 

Types 

Peacekeeping 

Foreign  Internal  Defense 

Security  Assistance 

Humanitarian  and  Civic  Assistance 
Support  to  Insurgencies 

Support  to  Counter  drug  Operations 
Combating  Terrorism 

Noncombatant  Evacuation  Operations 
Arms  Control 

Show  of  Force 

Support 

Operations 

Types 

Domestic  Support  Operations 

Foreign  Humanitarian  Assistance 

Forms 

Relief  Operations 

Support  to  Domestic  Consequence 
Management 

Support  to  Civil  Law  Enforcement 
Community  Assistance 

Several  approaches  have  been  used  for  attempting  to  quantify  the  operations  of  battlefield 
command  and  control  systems.  One  approach  is  the  function  analysis  methodology  (Ford, 
Mullen,  and  Keesling,  1997)  (from  a  previous  report  by  R.J.  Mullen  in  1996).  This  approach  has 
been  used  to  identify  systematic  structures  and  organization  of  the  tasks  critical  to  battlefield 
success.  Critical  command  and  control  tasks  were  identified  as  battlefield  functions  (BFs)  and 
were  organized  according  to  battlefield  operating  systems  (BOS).  Table  3  shows  these  task 
functions  and  their  applicability  to  battalion  and  brigade  TOCs.  The  task  functions  in  this  table 
are  another  view  of  the  task  list  presented  above. 
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Table  3  -  Battalion  and  Brigade  Battlefield  Functions  Grouped  by  BOS 
(Ford  et  al.,  1997) 


BOS 

Battlefield  Function  (BF) 

Applies 

To: 

Battalion 

Brigade 

Intelligence 

1.  Conduct  Intelligence  Planning 

X 

X 

2.  Collect  Information 

X 

X 

3.  Process  Information 

X 

X 

4.  Disseminate  Intelligence 

X 

X 

Maneuver 

5.  Conduct  Tactical  Movement 

X 

X 

6.  Engage  Enemy  with  Direct  Fire  and  Maneuver 

X 

Fire  Support 

7.  Employ  Mortars 

X 

8.  Employ  Field  Artillery 

X 

X 

9.  Employ  Close  Air  Support 

X 

X 

10.  Conduct  Electronic  Collection  and  Attack 

X 

11.  Conduct  PSYOP 

12.  Employ  Chemical  Weapons 

13.  Conduct  Counter  Target  Acquisition  Operations 

14.  Employ  Naval  Surface  Fires 

X 

15.  Coordinate,  Synchronize,  and  Integrate  Fire 

X 

X 

Support 

Air  Defense 

16.  Take  Active  Air  Defense  Measures 

X 

X 

17.  Take  Passive  Air  Defense  Measures 

X 

Command 

18.  Plan  for  Combat  Operations 

X 

X 

and  Control 

19.  Direct  and  Lead  Unit  During  the  Preparation  Phase 

X 

X 

of  the  Battle 

20.  Direct  and  Lead  Unit  in  Execution  of  Battle 

X 

X 

Mobility  and 

21.  Overcome  Obstacles 

X 

X 

Survivability 

22.  Enhance  Movement 

23.  Provide  Countermobility 

X 

X 

24.  Enhance  Physical  Protection 

X 

X 

25.  Provide  Operations  Security 

X 

X 

26.  Conduct  Deception 

27.  Conduct  NBC  Defense 

X 

X 

Combat 

28.  Provide  Transport  Services 

X 

X 

Service 

29.  Conduct  Supply  Operations 

X 

X 

Support 

30.  Provide  Personnel  Services 

X 

X 

31.  Maintain  Weapons  Systems  and  Equipment 

X 

X 

32.  Provide  Flealth  Services 

X 

33.  Treat  and  Evacuate  Battlefield  Casualties 

X 

X 

34.  Conduct  Enemy  Prisoners  of  War  Operations 

X 

35.  Conduct  Law  and  Order  Operations 

36.  Conduct  Civil  Affairs  Operations 

37.  Provide  Sustainment  Engineering 

38.  Evacuate  Non-combatants  from  Area  of  Operations 

39.  Provide  Field  Services 

The  effective  performance  of  critical  tasks  by  the  TOC  can  result  in  decisive  points  in  the 
battle.  Battlefield  decisive  points  are  defined  as  (Army  and  Marines,  1997;  BCTP,  2002): 
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•  A  point,  if  retained,  that  provides  a  commander  with  a  marked  advantage  over  his 
opponent.  Decisive  points  are  usually  geographic  in  nature,  but  could  include 
other  physical  elements  such  as  enemy  formations. 

•  A  time  or  location  where  enemy  weakness  is  positioned  that  allows  overwhelming 
combat  power  to  be  generated  against  it. 

•  Conveys  to  a  subordinate  a  potential  point  of  decision  that  a  commander  has 
identified  through  his  estimate  process  to  apply  overwhelming  combat  power. 

1.2.5.  The  State  of  C2  Analysis  and  Research. 

In  recent  years  there  has  been  an  increasing  volume  of  work  in  the  area  of  C2  as  its 
importance  continues  to  be  emphasized  but  its  meaning  and  ways  of  optimization  remain  vague. 
This  work  predominately  focuses  on  the  areas  of  individual  and  team  perfonnance,  mental 
workload,  decision  making  (including  the  Military  Decision  Making  Process,  MDMP), 
knowledge  elicitation,  to  name  just  a  few.  These  are  all  very  significant  topics  and  are  certainly 
worthy  of  continued  and  in  depth  research,  however,  while  the  literature  acknowledges  the 
existence  of  the  C2  system  there  is  very  little  on  going  effort  on  trying  to  describe  it  and  how  to 
address  C2  from  the  system  level.  The  reasons  are  obvious  as  while  human  performance  and 
workload  are  extremely  complicated  and  diverse  topics,  the  performance  of  a  military  C2  center 
is  even  more  so  by  at  least  an  order  of  magnitude.  This  is  why  it  is  felt  that  an  approach  like  the 
COMPASS  paradigm  has  such  a  great  potential  to  add  to  the  knowledge  base.  However,  there 
has  been  some  work  identified  that  addresses  various  components  of  the  C2  work  system  and  a 
few  of  them  are  summarized  here. 

1 .2.5.1 .  Operations  Other  Than  War  and  a  United  Kingdom  Land-Air  Simulation. 

One  approach  for  representing  the  whole  C2  process  in  simulation  was  attempted  for  an 
Operations  Other  Than  War  (OOTW)  simulation  and  a  United  Kingdom  (UK)  land-air  combat 
simulation  (Mason  and  Moffat,  2000).  The  simulation  represented  a  range  of  C2  operations  from 
top  down  rapid  planning  activities  to  bottom  up  deliberate  planning  activities.  It  attempted  to 
generically  describe  the  activities  the  C2  process  from  a  component  level  viewpoint.  Modeled  in 
the  simulation  were  the  commander,  communications  officer,  intelligence  officer,  force  planner, 
and  a  promulgator  that  disseminated  the  plan  to  subordinates.  The  model  used  the  recognition- 
primed  decision  (RPD)  model  (Klein,  1998)  of  the  decision  making  process  and  was  a 
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constructive  simulation  of  C2  in  combat  based  on  an  intelligent  agent  framework  of  C2  processes 
that  operated  in  accordance  with  simple  rules. 

1.2. 5.2.  Anny  Command  and  Control  Evaluation  System. 

The  U.S.  Army  Command  and  Control  Evaluation  System  (ACCES)  is  a  measurement 
system  developed  to  evaluate  the  effectiveness  of  C2  at  various  levels  (Halpin,  1992).  The 
premise  of  ACCES  is  that  C2  effectiveness  is  defined  as  the  effectiveness  of  the  headquarters 
staff  (synonymous  with  TOCs  at  battalion  and  brigade  levels  in  the  field).  This  program  was  a 
very  intensive  effort  to  evaluate  overall  performance  of  actual  TOCs  in  the  field.  It  employed  a 
team  of  data  collectors  to  collect  all  relevant  information  exchange  in  a  TOC  over  a  12-hour 
shift.  ACCES  is  a  bottom  up  manually  intensive  data  collection  technique  that  relies  on  both 
qualitative  and  quantitative  observation  in  the  TOCs  and  battle  simulation  center  (BSC)  for 
ground  truth.  Data  requirements  are  rigorous  and  the  data  collectors  were  required  to  complete  a 
14  lesson-training  program  before  being  sent  to  collect  data.  This  was  a  system  level  approach 
before  the  days  of  sophisticated  computer  simulation  that  employed  a  large  team  to  collect  data 
for  analysis  through  traditional  statistical  evaluation  techniques. 

1 .2.5.3.  Using  NNS  to  Evaluate  Mental  Workload  in  C2  Environments. 

One  study  was  conducted  to  test  the  predictions  of  a  mental  workload  model  (Hancock 
and  Caird,  1993).  As  it  was  noted  that  the  basic  premise  of  human  factors  is  a  primary  concern 
for  the  human  in  system  design  and  operation,  an  evaluation  of  a  mental  workload  model  was 
conducted  that  predicted  that  mental  workload  grew  as  perceived  distance  from  the  task  goal 
increases  and  the  effective  time  for  action  decreases.  This  study  encountered  the  same  problem 
found  in  many  other  studies.  This  problem  is  understanding  how  to  transform  the  composition  of 
performance  tasks  from  physical  to  cognitive  demands.  Here,  mental  workload  was 
conceptualizes  in  the  three  dimensions  of  effective  time  for  action,  perceived  distance  from  the 
desired  goal  state,  and  the  level  of  effort  required  to  achieve  the  desired  goal. 

Another  study  utilized  neural  network  simulation  models  to  predict  subjective  workload 
measures  using  condition  and  performance  measures  as  input  variables.  The  goal  was  in 
attempting  to  leam  how  condition  and  performance  factors  relate  to  differences  in  subjective 
workload  measures  in  TOC  structures  (Schvaneveldt,  Reid,  Gomez,  and  Rice,  1997).  Data  was 
collected  using  the  Subjective  Workload  Assessment  Technique  (SWAT)  (Reid,  Potter,  and 
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Bressler,  1986)  during  pursuit  tracking,  sequential  reaction  time,  and  tone  counting  tasks  to 
provide  workload  training  data.  The  NNS  models  were  trained  using  the  performance  data,  the 
condition  infonnation,  and  the  subjective  judgments  of  workload  from  the  SWAT  results.  The 
NNS  model  had  3  input  variables,  12  hidden  nodes,  and  one  output  node  representing  workload 
redline  condition.  Epoch  tests  went  from  less  than  100  to  over  100,000  with  training  achieved 
for  the  3-12-1  model  at  3020  epochs.  Study  results  indicated  that  linear  regression  models 
perform  as  well  as  nonlinear  neural  network  models  in  predicting  “redline”  workload  level  in 
training  data. 

A  recent  Ph.D.  dissertation  (Cioppa,  2002)  used  the  agent-based  simulation  MANA  to 
analyze  22  variables  in  a  complex  military  peace  enforcement  operation.  Here  a  set  of 
experimental  designs  were  developed  that  combined  orthogonal  Latin  Hypercubes  and  uniform 
designs  to  create  designs  having  near  orthogonality.  These  designs  were  used  to  develop  a 
capability  to  search  an  intricate  simulation  model  with  a  large  number  of  inputs  and  a  resulting 
complex  response  surface.  This  study  provided  an  interesting  approach  for  investigating 
command  system  perfonnance  while  trying  to  account  for  as  many  of  the  independent  measures 
as  possible.  The  paper  presented  a  solution  using  the  Latin  Hypercubes  for  models  of  up  to  22 
IV. 

Another  recent  study  (Jablunovsky  et  al.,  2000)  was  conducted  that  looked  at 
approximating  C2  network  behavior  using  a  neural  network  trained  on  the  relationships  between 
the  network  state  and  perfonnance.  Training  data  for  the  NNS  was  obtained  by  sampling  the 
surface  defined  by  network  latency  performance  as  a  function  of  the  C2  network  state  to  provide 
the  known  input  and  output  training  pairs  of  the  NNS  training  set.  The  training  algorithm 
employed  was  the  standard  iteratively  updated,  feed-forward,  error  backpropagation  algorithm. 
The  NNS  contained  41  C2  input  nodes,  30  hidden  nodes  in  one  layer,  and  5  output  nodes  with 
1250  training  pairs  and  125  validation  pairs.  The  intent  of  the  study  was  to  integrate  the  NNS 
into  a  two  sided  theater  level  simulation  called  THUNDER  with  the  objective  to  show  the  effects 
of  interdiction  and  degradation  of  the  C2  network  state  on  campaign  model  measures. 

1.2.6.  Conclusions  Regarding  C2  Analysis  and  Research. 

While  there  is  a  great  amount  of  research  and  investigative  work  into  C2  operations,  there 
is  little  aimed  at  the  overall  battlefield  management  system.  This  is  understandable  due  to  the 
complexities  of  trying  to  quantify  and  understand  individual  mental  workload,  let  alone  overall 
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system  performance.  The  application  of  the  COMPASS  paradigm  does  not  event  attempt  to 
address  internal  relationships  in  the  C2S,  it  simply  tries  to  quantify  it  through  the  identification 
of  significant  relationships  in  an  ever-growing  knowledge  base  that  drives  a  simulation  of  the 
TOC  processes.  One  of  the  big  challenges  in  this  performance  domain  is  the  transformation  the 
U.S.  Army  is  undergoing  with  the  conversion  to  digital  communication  processes.  It  is  not 
enough  to  model  the  current  state  of  the  system,  the  analysis  must  be  able  to  address  perceived 
future  systems.  This  is  one  reason  that  the  quick  turnaround  capabilities  of  modeling  from 
preexisting  knowledge  bases  has  merit.  There  are  many  obstacles  to  achieving  this  capability 
that  include  developing  an  understanding  of  the  true  nature  of  the  system  along  with  the  ability  to 
overcome  false  impressions  or  myths  about  it.  Some  of  the  current  myths  of  digital  technology 
are  (Lynch,  2001): 

1 .  TOCs  will  get  smaller  using  infonnation  technology. 

2.  Training  will  take  less  time. 

3.  We  need  “contractor  battalions”  to  support  us. 

4.  Digitization  will  show  us  an  immediate  impact  on  battlefield  operations. 

While  the  effectiveness  of  military  systems  will  continue  to  be  inextricably  linked  to  the 
performance  capabilities  of  human  operators  (Bowers  et  ah,  1997),  it  is  the  overall  perfonnance 
of  the  system  that  is  important.  A  common  characteristic  of  sub-system  components  in  current 
use  is  that  they  often  provide  vast  quantities  of  partially  relevant  data,  while  failing  to  identify 
the  information  which  the  decision  maker  actually  needs  to  solve  his  problem  (Cohen  and 
Freeling,  1981).  Some  of  the  issues  that  have  appeared  with  the  introduction  of  automated 
systems  into  C2  environments  are  (Corker  et  ah,  1990): 

1 .  Will  human  workload  saturate  a  particular  system  and  will  procedural 
bottlenecks  be  revealed? 

2.  What  will  the  duty  cycle  or  workload  of  an  operator  be  in  an  automated 
system? 

3.  What  is  the  impact  of  automation  initiatives  on  manpower  and  training  for 
new  systems? 

4.  What  will  be  the  effect  of  automation  on  the  information  and  data 
requirements  for  system  operation? 

5.  How  can  automation  be  effectively  transferred  into  the  existing  environment? 

6.  What  are  the  procedures  associated  with  system  verification  and  validation? 
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Finally,  it  is  noted  that  in  the  continuous  effort  to  improve  overall  C2  system 
performance  it  often  the  case  that  those  engaged  in  improving  human  perfonnance  seem  to  arrive 
on  the  scene  a  little  too  late  (Harper,  1974).  Analytical  tools,  like  COMPASS,  that  are  intended 
to  decrease  the  analytical  response  time,  is  one  approach  to  addressing  some  of  the  issues 
presented  here. 

1.3.  Approach. 

The  research  in  this  dissertation  was  conducted  in  two  phases.  Phase  I  was  be  the 
collection  of  data  from  TOC  operations  and  the  evaluation  of  it  with  datamining  techniques  such 
as  neural  networks  and  cluster  analysis.  The  neural  network  models  were  utilized  to  identify  and 
select  those  IV  that  are  significant  for  use  in  Phase  II.  Phase  II  took  the  descriptive  and 
numerical  models  developed  in  the  first  phase  and  applied  them  into  an  appropriate  pre-existing 
computer  simulation  of  TOC  operations. 

1.3.1.  Phase  I:  Data  Collection  Through  Naturalistic  Observation  Using  Ethnographic 

Techniques  and  Analysis  Using  Neural  Network  Simulation. 

The  three  TOC  deployments  observed  for  this  research,  which  are  depicted  in  Figure  3, 
provided  a  total  of  164  hourly  observations.  The  raw  data  from  the  exercises  were  standardized 
and  converted,  as  appropriate,  to  binary  for  evaluation  with  neural  network  simulation.  These 
data  consists  of  159  IV,  or  input  nodes  and  75  DV  or  target  nodes. 
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Figure  3  -  Exercise  Observations  /  Data  Collection 


These  three  TOC  observation  events  represent  only  the  start  of  what  is  envisioned  to  be  a 
very  large  dataset  containing  data  from  30,  40  or  more  TOC  deployments  over  the  next  few 
years.  Data  from  the  existing  observations  is  deemed  satisfactory  for  the  exploratory 
development  of  the  COMPASS  paradigm,  however,  it  falls  short  of  providing  satisfactory  depth 
for  the  purpose  of  substantiated  conclusions  regarding  TOC  performance  in  general.  It  has  been 
said  that  all  TOCs  are  different  while  all  TOCs  are  the  same.  This  statement  alludes  to  the  fact 
that  each  TOC  deployment  configuration  and  operational  configuration  is  dependent  upon  the 
personal  preferences  of  the  TOC  members  who  establish  its  layout  and  procedural  actions.  It 
also  means  that  all  similar  type  and  size  units  have,  in  essence,  the  same  basic  building  blocks  of 
hardware,  software,  and  tactical  equipment  with  which  to  set  up  the  TOC. 

As  the  TOC  observation  dataset  grows  to  include  information  from  a  dozen  and  more 
deployments  then  conclusions  can  start  to  be  generated  that  transcend  the  interpersonal 
differences  of  individual  opinion  for  setup  and  operation,  and  allow  the  analysis  to  concentrate 
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on  those  activities  and  constructs  that  are  germain  to  the  overall  configuration  and  operation  of 
TOCs  that  allows  for  the  most  efficient  and  successful  operation. 

The  exercise  data  was  evaluated  using  a  neural  network  simulation  called  Java  Neural 
Network  Simulation  (JNNS)  (Fischer,  Hennecke,  Bannes,  and  Zell,  2001)  developed  at  the 
University  of  Tubingen,  Wilhelm-Schickard  Institute  in  Tubingen,  Germany.  JNNS  runs  in  a 
Microsoft  Windows  environment  and  is  a  derivative  and  subset  of  a  more  complete  neural 
network  simulation  called  Stuttgart  Neural  Network  Simulation  (SNNS)  that  executes  on  Unix 
based  computers.  This  simulation  allows  the  development  of  neural  networks  that  can  be  trained 
and  pruned  to  reveal  relationships  to  be  identified  as  to  the  significance  of  input  nodes 
(corresponding  to  IV)  and  output  nodes  (corresponding  to  DV).  The  results  of  the  NNS  analysis 
are  used  to  provide  mathematical  models  of  the  relationships  of  the  IVs  to  the  DVs  for  use  as 
algorithmic  drivers  for  discrete  event  simulations  of  TOC  performance  in  Phase  II  of  this 
research. 

1.3.2.  Phase  II:  C2  Analysis  Using  Discrete  Event  Simulation. 

The  mathematical  models  developed  in  Phase  I  are  applied  to  discrete  event  simulations 
of  TOC  performance  developed  in  the  late  1990s  by  the  U.S.  Army  to  study  C2  organizations 
based  on  Army  needs  stemming  from  the  Persian  Gulf  war.  The  resulting  CoHOST  simulations 
examined  human  mental  and  physical  performance  capabilities  resulting  from  proposed  new 
digital  communications  systems.  The  CoHOST  model  architecture  was  developed  based  on  a 
taxonomy  of  human  performance  that  included  52  knowledge,  skills,  and  abilities  taxons 
organized  into  eight  categories  ranging  from  highly  physical  to  completely  cognitive  in  nature. 

A  CoHOST  design  was  implemented  with  these  taxonomic  based  descriptors  of  human 
performance  in  the  military  command  and  control  domain  using  the  commercial  programming 
language  Micro  Saint™.  For  this  research  the  results  of  the  JNNS  analysis,  in  the  form  of 
mathematical  models,  are  used  to  control  the  operation  of  the  CoHOST  model  to  optimize  its 
performance  according  to  the  observed  processes  in  real  TOCs. 

The  process  of  collecting  naturalistically  observed  data,  its  analysis  in  neural  network 
simulations  with  resulting  generation  of  mathematical  models  of  performance,  and  the  use  of 
these  models  to  tailor  the  performance  of  discrete  event  simulations  of  the  original  work  domain 
is  what  is  termed  the  COMPASS  paradigm  framework.  The  development  of  COMPASS  by  this 
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dissertation  is  intended  to  provide  a  research  basis  for  future  investigative  work  in  C2  work 
environments. 
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2.  Literature  Review. 


There  are  a  number  of  concepts  and  subject  areas  whose  understanding  and  current 
meanings  are  critical  to  the  development  of  the  research  presented  in  this  dissertation.  First,  a 
review  of  some  of  the  U.S.  Anny  techniques,  concepts,  and  procedures  (TCP),  that  have 
applicability  in  the  C2  domain,  are  presented  to  provide  a  conceptual  application  basis  for  this 
research.  Then,  there  are  a  number  of  research  areas  in  human  cognitive  performance  that  have 
significance  in  the  understanding  of  how  and  why  human  operators  tend  to  perform  the  way  they 
do  in  the  C2  arena.  Cognitive  task  analysis  procedures  and  concepts  form  the  basis  for  making 
observations  and  data  collection  to  support  the  research.  This  is  followed  by  a  review  of  some 
simulation  concepts  and  developmental  techniques  followed  by  a  review  of  some  analytical 
techniques  used  to  draw  conclusions  from  the  results  of  the  computer  simulations. 

2.1.  Selected  Military  Techniques,  Concepts,  and  Procedures. 

While  military  based  C2  can  be  considered  to  have  many  areas  of  commonality  with 
civilian  sector  counterparts,  it  is  unique  in  many  respects.  First,  approaches  to  military  C2  tend 
to  be  highly  procedural  concepts  that  are  well  documented  in  voluminous  staff,  technical,  and 
field  manuals.  Equipment  and  systems  that  support  military  C2  are  highly  specialized  that  are 
conceived  of  and  built  according  to  rigid  specification  designed  not  only  to  meet  performance 
needs,  but  also  to  have  an  ability  to  operate  in  widely  varying  location  and  climatic  conditions. 
Finally,  the  military  focus  on  mission  accomplishment,  especially  in  combat  conditions,  far 
exceeds  any  possible  counterpart  in  the  civilian  work  ethic. 

It  is  noted  that  the  U.S.  Army,  very  much  like  other  Federal  agencies  and  services,  is 
famous  or  infamous  for  its  development  and  use  of  acronyms  in  its  publications  and  normal 
operations.  To  the  uninitiated  this  can  become  daunting  while  trying  to  read  and  understand 
publications  containing  this  verbiage.  While  significant  effort  has  been  made  to  immediately 
identify  all  acronyms  used  in  this  dissertation,  there  may  be  some  omissions  as  well  as  references 
in  appendices  and  elsewhere  containing  terminology  that  seems  perfectly  nonnal  to  the  military 
reader  but  may  be  meaningless  to  others.  For  this  reason  a  glossary  of  tenninology  is  included 
that  has  been  compiled  from  official  Army  sources  (Army,  1993;  Army,  1998)  augmented  with 
tenns  from  various  Army  development  programs  such  as  the  Command  and  Control  Vehicle 
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(C2V),  the  Tactical  Unmanned  Aerial  Vehicle  (TUAV),  the  Force  XXI  Battle  Command  Brigade 
and  Below  system  (FBCB2),  and  the  Future  Combat  System  (FCS).  This  glossary  of  acronyms 
is  at  Appendix  B. 

The  TCPs  are  presented  in  a  logical  sequence  to  allow  an  understanding  of  concepts 
followed  by  techniques  and  procedures  (it  is  just  a  coincidence  that  they  are  in  alphabetical 
order.)  These  TCPs  are: 

2.1.1.  Anny  Operations  Orders. 

A  major  element  in  military  operations  orders  (OPORD)  is  the  commander’s  intent 
statement.  The  Commander’s  Intent  statement  helps  the  soldiers  read  the  commander’s  mind  if 
they  run  into  uncertainty  about  how  to  carry  out  the  orders  under  field  conditions  (Klein,  1998). 
One  description  of  the  commander’s  intent  statement  has  the  following  elements  (Weick,  1983): 

•  Here  is  what  I  think  we  face. 

•  Here’s  what  I  think  we  should  do. 

•  Here  is  why. 

•  Here  is  what  we  should  keep  our  eye  on. 

•  Now,  talk  to  me. 

Some  of  the  considerations  that  should  be  included  in  communicating  the  commander’s 
intent  are  (Klein,  1998): 

•  The  purpose  of  the  task. 

•  The  objective  of  the  task. 

•  The  sequence  of  steps  in  the  plan. 

•  The  rationale  for  the  plan. 

•  The  key  decisions  that  may  have  to  be  made. 

•  Anti-goals  or  unwanted  outcomes. 

•  Constraints  and  other  considerations. 

2.1.2.  Anny  Battle  Command  System. 

On  the  modem  battlefield  there  frequently  occurs  information  overload  conditions  where 
the  amount  of  information  may  reach  greater  quantities  than  can  be  assimilated  even  though  the 
number  of  communications  channels  employed  may  be  reduced.  The  realization  that 
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information  about  one’s  enemy  is  a  key  to  success  on  the  battlefield  is  not  a  new  one.  In  1832 
the  Prussian  General  Carl  von  Clausewitz  (Clausewitz,  1832)  wrote  in  his  classic  treatise 
concerning  the  nature  of  war  that  “By  the  word  “information”  we  denote  all  the  knowledge 
which  we  have  of  the  enemy  and  his  country;  therefore,  in  fact,  the  foundation  of  all  our  ideas 
and  actions.”  Concerning  the  nature  of  information  he  stated  that  the  “Great  part  of  the 
information  obtained  in  War  is  contradictory,  a  still  greater  part  is  false,  and  by  far  the  greatest 
part  is  of  a  doubtful  character.”  Some  technology  based  assistance  can  be  given  to  information 
acquisition  and  C2  operations,  however,  the  key  to  effective  C2  perfonnance  rests  on  the 
judgments  and  actions  of  key  personnel,  both  individually  and  collectively  (Olmstead, 
Christensen,  and  Lackey,  1973).  In  order  to  provide  the  best  technology  based  assistance  to 
these  key  decision  makers  in  the  field  the  Army  has  developed  and  is  in  the  process  of  fielding  a 
suite  of  digital  systems  to  provide  much  more  efficient  and  powerful  communications  systems 
that  greatly  increase  the  amount  of  infonnation  provided  to  military  decision  makers  without 
overloading  their  ability  to  assimilate  it. 

The  Army  Battle  Command  System  (ABCS)  was  designed  to  be  an  interoperable  system 
of  systems  using  a  client-server  architecture  on  a  local  area  network  (LAN)  to  achieve 
communications  interoperability  for  C2  systems.  The  ABCS  client-server  architecture  provides 
ability  for  any  ABCS  operator  to  access  the  data  and  view  the  displays  from  any  other  ABCS 
workstation  on  the  same  tactical  LAN.  The  ABCS  components  are  designed  to  answer  the 
following  six  questions  for  the  commander: 

•  Where  am  I? 

•  What  is  my  status? 

•  Where  are  the  other  friendly  units? 

•  What  is  their  status? 

•  Where  is  the  enemy? 

•  What  is  the  enemy’s  status? 

ABCS  provides  a  common  operational  picture  on  an  electronic  map  of  the  battlefield  that 
can  display  various  types  of  infonnation  such  as  friendly  unit  locations,  enemy  unit  locations, 
fire  support  control  measures,  air  and  tactical  ballistic  missile  tracks,  and  logistics  information 
(Army,  2002a). 
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Many  of  these  systems  and  their  interconnectivity  are  shown  in  Figure  4.  The  ABCS 
systems  include: 

•  GCCS-A  (Global  Command  and  Control  System).  This  is  a  corps  level  system  that 
provides  joint  infonnation  on  force  tracking,  host  nation,  civil  affairs,  theater  air  defense, 
targeting,  psychological  operations,  C2,  logistics,  medical,  and  personnel  status. 

•  MCS  (Maneuver  Control  System).  This  system  is  found  at  the  battalion  through  corps 
levels  and  provides  the  operations  staff  the  ability  to  monitor  the  current  battle  and  to 
plan  the  future  battle. 

•  MCS-L  (Maneuver  Control  System  -  Light).  This  system  is  a  reduced  version  of  MCS 
that  operates  on  a  Windows  NT  4.0  desktop  or  laptop  computer  system. 

•  ASAS  (All  Source  Analysis  System).  This  system  is  found  at  the  battalion  through  corps 
levels  and  receives  and  processes  enemy  information  from  national,  theater,  and  tactical 
echelons  to  include  FBCB2. 

•  ASAS-L  (All  Source  Analysis  System  -  Light).  This  system  is  a  reduced  version  of 
ASAS  that  operates  on  a  Windows  NT  4.0  desktop  or  laptop  computer  system. 

•  AFATDS  (Advanced  Field  Artillery  Tactical  Data  System).  This  system  is  positioned  at 
the  firing  battery  through  echelon  above  corps  (EAC)  levels  and  provides  the  artillery 
operations  staff  with  automated  fire  support  planning,  coordination,  and  control  of  close 
support,  counterfire,  interdiction,  suppression  of  enemy  air  defenses,  and  deep  operations. 

•  AMDPCS  (Air  and  Missile  Defense  Planning  and  Control  System).  AMDPCS  is  located 
at  the  air  defense  artillery  (ADA)  battery  through  EAC  levels  and  provides  the 
commander  with  the  ability  to  monitor  current  air  operations  while  planning  for  future 
events. 

•  FAADC  I  (Forward  Area  Air  Defense  Command,  Control,  Communications,  and 
Intelligence).  FAADC3I  provides  automated  interfaces  between  the  forward  area  air 
defense  nodes  and  weapon  systems 

•  CSSCS  (Combat  Service  Support  Control  System).  CSSCS  is  deployed  at  the  battalion 
through  corps  levels  and  provides  the  logistical  staff  with  the  ability  for  planning  and 
controlling  the  logistics  support  of  combat  operations. 

•  FBCB2  (Force  XXI  Battle  Command  -  Brigade  and  Below,  also  known  as  AB  -  Army 
Brigade  and  Below  Command  And  Control  System).  FBCB2  is  deployed  throughout  the 
battlefield  from  commander  to  platform  and  even  soldier  level.  It  provides  C2  and 
situational  understanding  (SU)  to  the  lowest  tactical  echelons  with  the  transmission  and 
receipt  of  orders,  reports,  and  data  via  combat  messages. 

•  TAIS  (Tactical  Airspace  Integration  System).  TAIS  is  found  at  the  division  through  EAC 
levels  and  is  a  system  that  provides  the  ground  commander  the  ability  to  take  part  in  the 
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management  of  the  air  battle.  It  is  the  interface  to  the  Army  Airspace  Command  and 
Control  (A2C2)  system  for  the  planning  and  operations  of  air  traffic  services. 

•  DTSS  (Digital  Topographic  Support  System).  DTSS  is  found  at  the  brigade  through 
corps  levels  and  enables  topographic  support  personnel  to  receive,  format,  store,  retrieve, 
create,  update,  and  manipulate  digital  topographic  data. 

•  IMETS  (Integrated  Meteorological  System).  IMETS  is  found  at  the  aviation  brigade  and 
the  division  and  corps  levels  providing  commanders  and  staffs  an  automated,  high- 
resolution  weather  system  to  receive,  process,  and  disseminate  current  weather 
observations,  forecasts,  and  environmental  effects  decision  aids. 

Five  of  these  systems  (MCS,  ASAS,  AFATDS,  AMDPCS,  CSSCS)  comprise  a  subset  of  the 
overall  ABCS  called  the  Army  Tactical  Command  and  Control  System  (ATCCS). 

MCS 


Figure  4  -  ABCS  Architecture 
(Adapted  From:  (Brown,  2002)) 

The  ABCS  systems  are  interconnected  by  a  tactical  communications  system  that  includes 
the  following  components: 

•  Mobile  Subscriber  Equipment  (MSE). 

•  Near  Term  Digital  Radio  (NTDR). 
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•  Single  Channel  Ground  and  Airborne  Radio  System  (SINCGARS). 

•  Enhanced  Position  Location  and  Reporting  System  (EPLARS). 

•  Wide  Area  Network  (WAN)  /  Local  Area  Network  (LAN)  switch  router  architecture. 

ABCS  provides  the  linkages  to  individual  soldiers  and  vehicles  to  the  overall  Army 
Tactical  Command  and  Control  System  (ATCCS).  ATCCS  links  together  all  of  the  functional 
area  control  systems  such  as  MCS,  FAADC3I,  CSSCS,  AFATDS,  and  FBCB2/AB2.  ATCCS 
connectivity  is  provided  by  the  Area  Common  User  System  (ACUS),  the  combat  net  radio 
(CNR)  systems,  and  the  Army  Data  Distribution  System  (ADDS)  (Anny,  1995).  The  overall 
ATTCS  architecture  is  shown  in  Figure  5. 


Figure  5  -  ATCCS  Architecture 
(Anny,  1995) 


2.1.3.  Battlefield  Operating  Systems  (BOS). 

Battlefield  Operating  Systems  are  groups  of  hardware  and  performance  tasks  that 
comprise  critical  tactical  activities  on  the  ground  battlefield.  The  BOS  provide  a  means  of 
reviewing  preparations  or  execution  in  discrete  subsets.  Army  BOSs  include  intelligence, 
maneuver,  fire  support,  mobility  and  survivability,  air  defense,  combat  service  support,  and 
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command  and  control.  These  are  illustrated  in  Figure  6.  Specifically  not  included  as  identified 
BOSs  are  intangible  activities  such  as  timing,  tempo,  reconnaissance,  information  operations, 
and  tactics  (Army  and  Marines,  1997).  BOS  are  defined  as  the  major  functions  that  occur  on  the 
battlefield.  These  are  performed  by  the  ground  forces  to  successfully  execute  battles  and 
engagements  to  accomplish  the  military  objectives  that  have  been  directed  by  the  ground  force 
commander.  The  seven  BOSs  are  described  below  (Gibbings,  1991).  Appendix  C  contains 
detailed  diagrams  showing  the  functions  for  each  BOS. 


Battlefield 

Operating 

Systems 

(BOS) 


Figure  6  -  Seven  Battlefield  Operating  Systems  (BOS) 

(Gibbings,  1991) 

•  The  Maneuver  BOS  is  the  employment  of  forces  on  the  battlefield  through 
movement  and  direct  fires.  It  is  supported  by  fire  support  artillery  and  has  the 
goal  of  achieving  a  position  of  advantage  over  the  enemy  to  accomplish  the 
mission. 

•  The  Fire  Support  BOS  is  the  collective  and  coordinated  use  of  target  acquisition 
data  by  indirect  fire  weapons,  armed  aircraft  (excluding  armed  helicopters),  and 
other  lethal  and  nonlethal  means  against  ground  targets.  This  BOS  is  always 
employed  in  support  of  maneuver  force  operations.  Indirect  fire  weapons 
includes  tube  artillery,  rocket  and  missile  artillery,  mortars,  naval  gunfire,  close 
air  support,  electronic  countermeasures,  and  all  other  non-line-of-sight  fires. 
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•  The  Air  Defense  BOS  includes  all  measures  designed  to  nullify  or  reduce  the 
effectiveness  of  attack  by  hostile  aircraft  or  missiles  after  they  are  airborne.  This 
BOS  includes  all  weapon  systems  with  the  potential  to  engage  aerial  targets. 

•  The  Command  and  Control  BOS  is  the  exercise  of  authority  and  direction  by  a 
commander  over  subordinate  forces  to  achieve  mission  accomplishment.  C2 
functions  are  perfonned  through  an  arrangement  of  personnel,  equipment, 
facilities,  and  procedures  employed  by  the  commander  and  the  staff  in  the 
planning,  directing,  coordinating,  and  controlling  of  forces  and  operations  to 
achieve  mission  accomplishment. 

•  The  Intelligence  BOS  is  the  collection  of  activities  that  generate  knowledge  of  the 
enemy,  weather,  and  geographical  features  required  by  the  commander  for 
planning  and  conducting  combat  operations.  This  BOS  is  derived  by  analyzing 
information  on  the  enemy’s  capabilities,  intentions,  vulnerabilities,  and  the 
environment. 

•  The  Mobility  and  Survivability  BOS  describes  the  functions  of  the  ground  force 
that  permits  freedom  of  movement  relative  to  the  enemy  while  retaining  the 
ability  to  fulfill  the  primary  mission.  It  also  includes  those  measures  taken  to 
remain  viable  and  functional  by  achieving  protection  from  the  effects  of  enemy 
weapons  as  well  as  natural  occurrences. 

•  The  Combat  Service  Support  BOS  is  the  support  and  assistance  provided  to  the 
ground  force  to  sustain  it  in  the  fields  of  logistics,  personnel  services,  and  health 
services.  This  BOS  describes  activities  that  may  be  provided  by  the  U.S.  Army, 
the  host  nation,  or  by  contracted  support  to  man,  arm,  fuel,  fix,  and  move  the 
Anny  ground  forces  during  times  of  combat  operations.  It  also  includes  the 
functions  to  build  and  maintain  facilities  and  provide  military  police  support. 

Military  tactical  units  are  excellent  examples  of  organizations  that  must  adapt  readily  to 
fast  changing  environmental  conditions  (Olmstead  et  ah,  1973).  The  ABCS  system  is  an  integral 
component  of  the  Army’s  Objective  Force  (Army,  2002b)  program  development  strategy. 

2.1.4.  Command  and  Control. 

It  has  been  stated  that  there  is  nothing  more  important  to  success  on  the  battlefield  than 
effective  C2  (Goedkoop,  1988).  The  United  States  military  in  general  and  the  U.S.  Army  in 
particular  have  an  enormous  investment  in  the  expenditure  to  study,  analyze,  quantify,  and 
understand  the  concept  of  C2.  Even  today,  after  more  than  50  years  of  intense  effort  in  the  post 
World  War  II  era  this  topic  and  process  remains  an  enigma  that  is  the  subject  of  constant  study, 
analysis,  and  debate.  The  new  operational  paradigm  that  is  promulgating  throughout  the  Army 
as  a  result  of  the  decision  to  digitize  the  force  has  only  served  to  heighten  the  intensity  with 
which  this  subject  is  addressed.  Military  tactical  environments,  that  are  both  turbulent  and 
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unpredictable,  are  characteristic  of  the  present  and  anticipated  future  environments  that  do  and 
will  require  C2  functions  to  respond  flexibly  to  a  more  or  less  constant  flow  of  situations 
characterized  by  uncertainty  (Olmstead  et  ah,  1973).  The  advent  of  digitization  of  C2  has  caused 
a  change  in  the  way  that  operators  in  these  environments  operate  as  they  are  mainly  faced  with 
performing  tasks  related  to  information  processing  (Schaafstal  and  Schraagen,  1992).  The 
proliferation  of  this  information  technology  has  spawned  a  dire  need  for  the  ability  to  measure 
and  evaluate  system  perfonnance  in  a  comprehensive  manner  (Kleiner,  1997). 

The  U.S.  defense  establishment  has  manuals,  regulations,  and  publications  on  every 
imaginable  topic  and  sources  for  formal  definitions  of  command  and  control  are  no  exception. 
Some  of  these  authoritative  definitions  are  (Huron,  1997;  JCS,  1994): 

•  Coimnand  -  The  authority  that  a  coimnander  in  the  Anned  Forces  lawfully 
exercises  over  subordinates  by  virtue  of  rank  or  assignment.  Battle  coimnand  is 
the  application  of  leadership  as  an  element  of  combat  power.  It  involves  four 
functions:  visualizing,  describing,  directing,  and  leading.  It  is  the  exercise  of 
coimnand  in  operations  against  a  hostile,  thinking  enemy  (Army,  2001c).  It  is 
also  described  as  the  art  of  battle  decision  making  and  the  leading  and  motivating 
of  soldiers  and  their  organizations  into  battle.  It  is  the  natural  expansion  of  C2  as 
a  result  of  changes  in  the  scope  and  intensity  of  current  operations  and  the 
prospect  of  future  operations  (Anny,  1996). 

•  Control  -  Authority  which  may  be  less  than  full  command  exercised  by  a 
coimnander  over  part  of  the  activities  of  subordinate  or  other  organizations. 

•  Coimnand  and  Control  System  (C2S)  -  The  facilities,  equipment, 
communications,  procedures,  and  personnel  essential  to  a  commander  for 
planning,  directing,  and  controlling  operations  of  assigned  forces  pursuant  to  the 
missions  assigned.  However,  it  is  not  just  the  arrangement  of  equipment  such  as  a 
communications  system.  The  concept  of  a  C2S  is  that  it  is  the  personnel, 
equipment,  communications,  facilities  and  procedures  that  facilitate  the 
commander ’s  ability  to  command  and  control.  The  C2S  is  the  organization  of 
resources  available  to  the  commander  to  help  plan,  direct,  coordinate,  and  control 
military  operations  within  the  unit.  The  goal  is  the  assurance  of  mission 
accomplishment  (Army,  1997).  It  is  also  the  brain  of  a  tactical  unit,  collating  all 
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information  and  sending  appropriate  instructions  to  personnel  who  are  in  contact 
with  the  enemy.  The  extent  to  which  this  system  functions  flexibly,  efficiently, 
and  effectively  detennines  the  ability  of  the  unit  to  succeed  on  the  battlefield 
(Olmstead  et  ah,  1973). 

•  Command  and  Control  -  The  exercise  of  authority  and  direction  by  a  properly 
designated  commander  over  assigned  and  attached  forces  in  the  accomplishment 
of  the  mission  (Army,  1997). 

Another  definition  of  the  C2S  is  that  it  is  comprised  of:  (Pendergrass  and  Hughes,  1993): 

•  Physical  elements  -  transmitters,  signal  lights,  computers,  codebooks,  tapes, 
deciphering  equipment,  etc. 

•  Human  elements  -  the  commander,  staff,  military  analysts  in  the  chain  of 
command,  etc. 

•  Procedural  elements  -  used  to  conduct  the  process  including  training  manuals, 
equipment  manuals,  organization  charts,  and  command  relationships. 

Command  and  control  is  described  as  having  two  components.  These  are  the  commander 
and  the  C2S.  Subsystems  to  the  C2S  include  communications,  intelligence,  and  computer 
networks,  which  provide  the  backbone  for  the  overall  C2S.  The  effective  use  of  these  systems 
and  subsystems  is  what  allows  the  commander  the  ability  to  lead  from  any  point  on  the 
battlefield  (Army,  2001c). 

The  U.S.  Army  and  U.S.  Marine  Corps  formally  define  C2  as  (Army  and  Marines,  1997): 

“The  exercise  of  authority  and  direction  by  a  properly  designated  commander  over 
assigned  and  attached  forces  in  the  accomplishment  of  the  mission.  Command  and  control 
functions  are  perfonned  through  an  arrangement  of  personnel,  equipment,  communications 
facilities,  and  procedures  employed  by  a  commander  in  planning,  directing,  coordinating,  and 
controlling  forces  and  operations  in  the  accomplishment  of  the  mission.” 

Procedures  for  effective  C2  can  be  highly  individualistic  and  interactive  and  are  therefore 
“not  cut  and  dried”.  Battlefield  C2  is  described  as  a  constant  balancing  act  where  the  need  for 
detailed  planning  must  be  balanced  against  the  need  for  quick  and  decisive  action  (Fallesen  et  ah, 
1992a).  A  symbolic  model  of  these  activities  is  at  Figure  7.  Tactical  command  and  control  is 
made  up  of  three  primary  activities: 
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•  Planning  -A  continuous  activity  conducted  to  prepare  for  assigned  or  assumed 
tasks. 

•  Directing-The  commander  decides  on  a  particular  action  and  then  directs  its 
execution. 

•  Monitoring-The  process  of  discerning  what  is  happening  and  what  it  means. 


Figure  7  -  Model  of  Command  and  Control  Process 
(Fallesen  et  ah,  1992a) 


Another  accepted  model  of  command  and  control  process  is  the  Lawson  Loop.  This 
model  was  developed  by  Joel  S.  Lawson  and  is  considered  applicable  to  C2  processes  ranging 
from  the  Napoleonic  era  to  the  1990s.  This  model  consists  of  five  functions  (Huron,  1997): 

•  Sense  -  the  collection  of  data  from  the  environment. 

•  Process  -  pulls  together  and  correlates  the  data  gathered  from  the  sense  function. 

•  Compare  -  comparison  of  the  information  just  processed  in  the  process  function 
to  the  current  state  of  the  environment  to  the  desired  state  of  the  environment. 

•  Decide  -  decisions,  performed  normally  by  the  commander,  from  the  courses  of 
action  presented  by  the  commander’s  staff. 

•  Act  -  the  manifestation  of  the  commander’s  decision. 

This  model  is  illustrated  in  Figure  8. 
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Figure  8  -  The  Lawson  Command  and  Control  Loop 
(Huron,  1997) 


Evaluations  of  C2,  C3,  and  C4  systems  all  need  to  be  performed  with  the  aim  to  establish 
the  extent  to  which  the  system  will  support  the  effective  prosecution  of  combat  operations.  It 
should  also  be  to  answer  the  question  of  whether  an  investment  in  a  more  capable  system  would 
result  in  more  combat  effectiveness  than  an  equal  investment  in  alternatives,  including  the 
fighting  forces  themselves  (Snyder,  1993). 

2.1.5.  T actical  Operations  Center. 

A  TOC  is  that  group  of  personnel  and  equipment  used  by  combat  unit  headquarters  to 
control  operations  on  a  battlefield.  TOCs  for  battalion  and  brigade  level  units  tend  to  be  similar 
as  they  use  the  same  type  of  equipment  and  general  organization.  TOCs  for  division  and  corps 
level  units  are  also  similar  in  nature  but  are  significantly  bigger  and  more  complex  as  these  levels 
of  headquarters  exercise  authority  over  a  much  larger  sector  of  the  battlefield.  An  example  of  a 
brigade  TOC  layout  that  was  actually  used  during  the  Battle  Command  Training  Program 
(BCTP)  Warfighter  computer  simulation  wargame  exercise  Phantom  Thunder  conducted  at  Fort 
Hood,  Texas  in  February  of  2002  is  shown  in  Figure  9.  BCTP  is  a  training  program  based  in 
Fort  Feavenworth,  Kansas  that,  along  with  the  National  Training  Center  (NTC)  at  Fort  Irwin, 
California,  evaluates  units  on  their  warfighting  capabilities.  BCTP  performs  their  evaluations 
through  the  use  of  computer  wargame  simulations  and  the  NTC  performs  its  evaluations  through 
live  field  exercises  conducted  against  live  opposing  force  (OPFOR)  units. 
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This  TOC  layout  was  used  by  the  1st  Brigade  of  the  1st  Cavalry  Division  during  this 
exercise  (BCTP,  2002).  The  individual  sections  such  as  the  S3,  ADA,  and  S2  are  fonned  around 
Ml 068  tracked  command  post  vehicles.  The  area  in  the  center  between  the  Ml 068s  is  formed 
using  tactical  tents  interconnected  together  and  to  the  vehicles. 
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Figure  9  -  TOC  Layout  for  1st  Bde,  1CD  During  Exercise  Phantom  Thunder 

The  officers  assigned  to  the  TOC  and  their  enlisted  assistants  frequently  suffer  from 
viscerogenic  deficiencies,  especially  the  gross  lack  of  rest,  incomplete  diet,  and  often 
overwhelming  psychological  influences  resulting  from  the  very  nature  of  combat  operations.  In 
addition  to  functioning  as  a  control  center,  the  TOC  frequently  is  placed  under  bombardment  or 
ground  attack  by  the  enemy  forces  which  requires  that  the  personnel  assigned  to  the  command 
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center  be  able  to  render  judgments  and  decisions  while  engaged  in  close  combat,  repelling  the 
enemy,  and  often  while  attempting  to  destroy  classified  documents  and  equipment.  Military 
personnel  employed  within  the  TOC  are  required  to  accumulate  a  sizeable  number  of  facts, 
relationships,  and  hypotheses  while  being  constantly  conversant  about  the  friendly  and  enemy 
force  situations  and  intentions.  It  is  within  the  TOC  that  the  unit  commander,  operations  officer, 
intelligence  officer,  and  supply  officer  direct  and  support  the  operations  of  the  command.  The 
essence  of  success  in  this  role  is  to  remain  in  control  of  one’s  emotions  and  faculties  while 
responding  to  one’s  training  (Gordon,  1973). 

Effective  operation  of  the  TOC  is  tied  to  its  ability  to  ensure  effective  C2  within  the 
battalion/task  force  and  brigade/brigade  combat  team  organizations  (Goedkoop,  1988).  The  key 
to  doing  this  is  the  successful  integration  of  the  commander’s  tactical  intent  throughout  all 
phases  of  the  TOCs  tactical  operations.  These  tactical  operational  requirements  can  be  tied  to  the 
seven  BOS  systems.  TOC  C2  requirements  organized  according  to  BOS  requirements  are  shown 
in  Table  4. 

Table  4  -  TOC  C2  Functional  Requirements  According  to  BOS 

(Goedkoop,  1988) 


BOS 

TOC  Performance  Tasks 

1.  Maneuver  of  combat 
elements  of  the  unit. 

•  Recommends  new  maneuver  courses  of  action  to  the  commander  when 
the  situation  dictates. 

•  Tracks  movement  of  maneuver  elements  in  accordance  with  the  tactical 
plan. 

•  Analyzes  unit  reporting  to  ensure  execution  in  accordance  with  the 
tactical  plan;  adjusts  as  necessary. 

•  Tracks  movement  of  adjacent,  higher,  and  rear  units. 

•  Integrates  attack  helicopters  into  maneuver  scheme. 

2.  Intelligence  that 

provides  the  ability  to 
“see”  the  battlefield. 

•  Maintains  enemy  order  of  battle  working  map. 

•  Continues  to  analyze  /  develop  enemy  situation  during  the  battle. 

•  Provides  periodic  update  of  enemy  situation  to  commanders/maneuver 
elements. 

•  Passes  intelligence  to  higher  and  adjacent  units. 

•  Requests  situation  reports/battle  damage  assessments  from  unit  elements 
aggressively. 

•  Revises  template  as  information  is  received. 

•  Continues  named  area  of  interest  surveillance  during  the  battle; 
repositions  collection  assets  as  required. 

•  Passes  enemy  unit  locations  to  the  fire  support  element  for  targeting. 

•  Advises  commander  on  decision  points  as  reached. 

•  Adjusts  enemy  movement  time  lines  as  needed. 

•  Updates  weather  and  wind  direction,  assists  S3  and  chemical  officer  with 
planning  for  use  of  smoke. 
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BOS 

TOC  Performance  Tasks 

•  Advises  on  possibility  of  enemy  nuclear,  biological,  or  chemical  weapon 
usage. 

•  Maintains  record  of  enemy  losses  by  type  of  vehicle  to  assist  in 
templating/order  of  battle. 

•  Maintains  status  of  task  force  collection  assets. 

•  Maintains  ground  surveillance  radar  and  remote  sensor  locations  and 
adjusts  as  necessary. 

•  Monitors  counter-reconnaissance  battle. 

•  Requests  and  integrates  information  from  higher/adjacent  units. 

•  Processes  shell,  bomb,  and  mortar  reports. 

•  Operates  unit  operations/intelligence  radio  net  as  required. 

•  Screens  information  from  enemy  prisoners  of  war. 

3.  Fire  Support  that 
masses  firepower  to 
delay,  disrupt,  or 
destroy  enemy  forces 
in  support  o  f  the 
scheme  of  maneuver. 

•  Maintains  maneuver  unit  front-line  trace;  provides  to  the  direct  support 
artillery  unit  and  the  higher  fire  support  element. 

•  Clears  indirect  fires  within  the  unit’s  sector/zone  of  operations. 

•  Reports  changes  of  priorities  of  fire/final  protective  fires  to  the  field 
artillery  support  unit. 

•  Coordinates  for  placement  of  field  artillery  supporting  units  in 
sector/zone  of  maneuver  unit. 

•  Ensures  that  the  fire  support  plan  is  executed  per  the  fire  support  matrix. 

•  Updates  fire  plans  as  necessary-  plans  for  contingencies,  new  enemy 
locations. 

•  Adjusts  fire  coordination  measures  as  required. 

•  Engages  targets  of  opportunity  as  directed  by  the  Executive  Officer/S3 

Air. 

•  Maintains  communications  with  the  mortar  platoon,  field  artillery  fire 
direction  center,  and  fire  support  teams  (FISTs). 

•  Updates  FISTs  on  the  status  of  fire  support  availability. 

•  Relays  FIST  calls  for  fires  to  direct  support  battalion  when  FIST  cannot 
do  so. 

•  Ensures  mortars  are  integrated  into  the  fire  support  plan;  Recommends 
mortar  repositioning  as  necessary. 

•  Provides  current  artillery  ammunition  status  to  maneuver  unit. 

•  Plans  with  the  engineer/  executes  artillery  delivered  family  of  scatterable 
mines  (FASCAM). 

•  Coordinates  close  air  support  missions. 

•  Ensures  airspace  coordination  areas  are  put  into  effect  and  cancelled  as 
needed. 

•  Plans  and  requests  suppression  of  enemy  air  defense  missions  to  support 
close  air  support  /  joint  air  attack  team  strikes. 

•  Coordinates  requests  for  additional  fire  support. 

4.  Combat  Service 

Support  actions  taken 
to  sustain  the  unit’s 
ability  to  fight. 

•  Ensures  combat  trains  command  post  tracks  friendly  situation/status. 

•  Ensures  units  are  reporting  losses  via  Administrative/Logistics  radio 
nets. 

•  Maintains  status  of  critical  supply/ammunition  items. 

•  Maintains  status  of  unit  combat  power/personnel  losses. 

•  Informs  S4  when  to  displace  combat  trains. 

•  Maintains  location  of  key  logistical  facilities;  Field  trains,  combat  trains, 
unit  maintenance  collection  point. 

•  Coordinates  time/location  of  unit  logistical  package  (LOGPAC)  arrival. 

•  Directs  priorities  of  movement  on  task  force  supply  routes. 

5.  Air  Defense  system 

•  Updates  air  defense  warnings/weapons  control  status. 
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BOS 

TOC  Performance  Tasks 

providing  security 
from  enemy  close  air 
support  aircraft  and 
attack  helicopters. 

•  Disseminates  “Red  Air”  early  warning  over  command  radio  net. 

•  Announces  friendly  aircraft  arrival  on  station. 

•  Employs  passive  and  active  air  defense  measures  as  required. 

•  Disseminates  “Red”  air  assault  operations  in  sector. 

•  Monitors  status  of  air  defense  weapons/units;  adjusts  coverage  as 
necessary  based  on  combat  losses  or  new  contingencies. 

•  Ensures  air  defense  elements  maintain  location/situation  of  task  force 
lead  elements. 

•  Assists  communications  between  Vulcan/Stinger  assets  and  parent 
organizations. 

6.  Mobility, 

Countermobility, 
Survivability  that: 

-  Preserves  freedom  of 

maneuver  of  friendly 
forces; 

-  Obstructs  maneuver  of 

the  enemy; 

-  Enhances  survivability  of 

friendly  forces. 

•  Tracks  execution  status  of  Countermobility/mobility/survivability  plan — 
reports  to  higher  headquarters. 

•  Monitors  status  of  bulldozers,  armored  combat  earthmovers,  small 
emplacement  excavators/class  III  resupply. 

•  Tracks  status  of  Class  IV/V  barrier  stocks  and  distribution. 

•  Ensures  task  force  supply  route  remains  passable  for  combat  service 
support  assets. 

•  Reports  obstacles,  breaches  or  gaps  in  enemy  obstacles  to  subordinate 
and  higher  units. 

•  Anticipates  requirements  to  smoke/suppress  enemy  when  task  force  hits 
obstacles. 

•  Continues  FASCAM  planning  during  operations  to  deny  flanks/slow 
enemy  movement. 

•  Upgrades  mission  oriented  protective  posture  status  as  required. 

•  Maintains  unit  radiological  exposure  status. 

•  Analyzes,  prepares,  and  disseminates  nuclear,  biological  and  chemical 
reports  as  required. 

•  Advises  the  second-in-command  on  unmasking  procedures. 

•  Coordinates  for  decontamination  support  as  required. 

•  Maintains  effective  downwind  message. 

•  Ensures  TOC  locations  provides  survivability/camouflage. 

•  Monitors  operations  security  program  during  battle. 

•  Ensures  local  security/air  guard  of  TOC  is  maintained. 

7.  Command  and  Control 
system  that  plans, 
coordinates,  and 
executes  combat 
operations. 

•  Issues  fragmentary  orders. 

•  Maintains  radio  communications  with  higher,  adjacent,  and  subordinate 
headquarters. 

•  Maintains  net  discipline  as  command  net  control  station. 

•  Communicates  with  subordinates  the  commander  or  S3  cannot  reach; 
relays  as  necessary. 

•  Monitors  critical  subordinate  command  nets  during  contact. 

•  Initiates  frequency  changes  if  jammed;  polices-up  stations  from  old  net  if 
necessary. 

•  Activates  “Battle  Net”  if  key  elements  within  the  task  force  lose  secure 
capabilities. 

•  Maintains  accurate  status  charts  for  critical  information  items. 

•  Manages  synchronization  matrix  in  accordance  with  tactical  plan. 

•  Disseminates  tactical  information  to  subordinate  units. 

•  Displaces  TOC  during  operations  to  facilitate  command  and  control. 

•  Ensures  subordinates  continue  to  report  during  the  conduct  of  operations. 

•  Second-in-Command  responds  for  commander  on  higher  command  net 
to  allow  the  commander  to  fight  the  battle. 

•  Requests  and  coordinates  additional  combat/combat  support  assets  from 
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TOC  Perfonnance  Tasks 

brigade. 

•  Provides  routine  or  requested  situation  updates/reports  to  higher 
headquarters. 

•  Lays  land  line  to  combat  trains  command  post  when  possible. 

•  Manages  terrain  in  unit  sector. 

•  Initiates  planning  for  future  operations. 

As  a  result  of  hundreds  of  battalion  and  brigade  TOCs  that  have  trained  at  the  National 
Training  Center  (NTC)  at  Fort  Irwin,  California,  there  have  been  countless  insights  on  how 
TOCs  function  during  combat  operations.  These  experiences  provide  insights  into  problem  areas 
that  are  systemic  in  nature  to  TOC  operations  (at  least  in  the  U.S.  Army)  and  transcend 
individual  problems  and  deployments.  Some  of  these  noted  weaknesses  are  (Goedkoop,  1988): 

•  Inability  of  the  TOC  to  track  the  flow  of  the  battle,  and  synchronize  the  actions  of  the 
task  force. 

•  The  performance  of  the  TOC  is  generally  unsatisfactory  when  the  executive  officer  does 
not  operate  from  it  during  combat  operations. 

•  A  displacement  plan  is  seldom  produced  to  guide  the  movement  of  the  TOC  in 
consonance  with  the  tactical  plan  with  the  result  that  the  TOC  often  is  left  behind  and 
loses  communication  with  the  lead  elements. 

•  Local  security  and  small  anns  air  defense  protection  of  the  TOC  are  often  neglected 
during  combat  operations. 

•  If  the  TOC  is  destroyed  the  alternate  TOC  (normally  the  combat  trains  command  post 
(CTCP))  must  cease  its  nonnal  duties  and  assume  the  duties,  albeit  in  a  degraded  mode, 
of  the  TOC  with  the  result  that  a  rippling  effect  goes  throughout  the  entire  organization 
causing  the  effectiveness  of  the  whole  organization  to  suffer. 

•  TOCs  have  an  obvious  visual  signature  due  to  the  large  number  of  vehicles,  which 
normally  compose  it.  This  often  causes  it  to  be  targeted  and  subsequently  destroyed  by 
high  perfonnance  aircraft. 

•  Most  TOCs  do  not  practice  adequate  communications  security.  The  large  number  of 
radio  nets  normally  operated  at  the  TOC  produce  an  electronic  signature  that  is  subject  to 
exploitation  by  electronic  means  such  as  direction  finding  equipment  and  jammers. 

2.1.6.  MANPRINT. 

It  was  not  until  World  War  II  that  the  U.S.  Department  of  War  recognized  the  importance 
of  designing  systems  that  meet  human  perfonnance  requirements  (O’Brien,  1985).  Beginning  in 
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the  early  1980’s  with  a  realization  of  this  requirement,  the  U.S.  Army  acknowledged  that 
developing  and  fielding  the  right  kind  of  force  in  the  face  of  continued  technological 
advancement  combined  with  evolving  changes  in  the  strategic  environment  is  a  big  challenge 
(Army,  2001b).  A  predominate  issue  is  the  integration  of  human  performance  abilities  into  the 
system  design  and  acquisition  process.  The  Army  program  to  incorporate  human  factors  issues 
into  its  materiel  acquisition  and  development  process  is  called  Manpower  and  Personnel 
Integration  (MANPRINT). 

MANPRINT  is  mandated  by  Anny  Regulation  (Army,  1991)  and  is  a  management  and 
technical  effort  to  integrate  soldier  performance  and  reliability  issues  into  the  acquisition  process 
and  is  a  comprehensive  technical  effort  to  promote  system  effectiveness  by  integrating  into  the 
materiel  development  and  acquisition  process  all  relevant  information  concerning  human  factors 
engineering,  system  safety,  and  health  hazards  (Barber,  Ching,  Jones,  and  Miles,  1990;  Guerrier, 
Lowry,  Jones,  Guthrie,  Barber,  and  Miles,  1991).  MANPRINT  is  the  Army’s  execution  of 
DoD’s  Human  systems  Integration  (HSI).  DOD  5000.2-R,  Part  Four,  Paragraph  4.3.8,  states,  “A 
comprehensive  management  and  technical  strategy  shall  be  initiated  early  ...  to  ensure  that 
human  perfonnance  ...  is  considered  throughout  the  system  design  and  development  process 
(Walker,  1997).”  It  is  an  Anny  program  developed  to  optimize  the  human  dimensions  of  system 
design  through  early  involvement  in  systems  design  /  systems  engineering  (Thurmond  and 
Collins,  1988).  It  is  a  comprehensive  management  and  technical  program  to  enhance  human 
performance  and  reliability  in  the  operation,  maintenance  and  use  of  weapon  systems  and 
equipment. 

The  total  MANPRINT  system  includes  all  of  the  people,  equipment,  doctrine,  training, 
etc.,  necessary  to  field  and  sustain  the  system  in  peacetime  and  combat.  The  goal  of 
MANPRINT  is  to  optimize  total  system  perfonnance  by  influencing  the  design  and  fielding  of 
new  weapon  and  other  systems  to  improve  battlefield  effectiveness  and  reduce  operations  and 
support  costs  through  the  continuous  integration  of  human  factors  engineering,  manpower, 
personnel,  training,  system  safety  and  health  hazard  considerations  throughout  the  materiel 
development  and  acquisition  process(Blackwood  and  Dice,  1988;  Bogner,  1989;  Lickteig,  1987). 
Some  of  the  so-called  “MANPRINT  Rules  of  Thumb”  are: 

1 .  Soldier  performance  affects  system  performance. 

2.  Aptitude  plus  training  =  skill. 
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3.  Measure  soldier  performance  by  time  and  accuracy. 

4.  Equipment  design  determines  soldier  tasks. 

5.  Make  the  designer  responsible  for  soldier  performance. 

MANPRINT,  in  practice,  is  the  recognition  that  the  capabilities  and  limitations  of  the 
soldiers  who  operate,  maintain,  and  support  Army  equipment  must  be  an  important  consideration 
when  designing  or  selecting  hardware.  The  MANPRINT  process  refers  to  those  specific  actions 
that  must  be  accomplished  to  ensure  that  soldier  performance  issues  are  identified,  addressed, 
and  managed  throughout  the  design,  development,  and  acquisition  of  a  new  materiel  system 
(Johnson  and  Wright,  1990).  MANPRINT  is  a  comprehensive  management  and  technical 
program  that  focuses  attention  on  human  capabilities  and  limitations  throughout  the  systems  life 
cycle  including  concept  development,  documentation,  design,  development,  fielding,  operation 
and  modernization  of  systems  (Walker,  1997).  The  MANPRINT  program  is  a  comprehensive 
management  and  technical  program  to  assure  total  system  effectiveness  by  continuous 
integration  into  materiel  development  and  acquisition  of  all  relevant  infonnation  concerning 
Manpower,  Personnel,  Training,  System  Safety,  Health  Hazards,  and  Human  Factors  engineering 
(Hiemstra,  Korzym,  Barila,  and  Imbs,  1987;  Johnson,  Rossmeissl,  Kracov,  and  Shields,  1988). 

The  search  for  decisive  weapons  has  led  to  a  continued  high  level  of  investment  in 
advanced  technologies.  The  implementation  of  well  conceived,  integrated  HFE  programs  is 
intended  to  mitigate  or  resolve  many  of  the  identified  human  performance  deficiencies  in  the 
design  of  military  systems  (Price,  Sawyer,  and  Kidd,  1983).  During  the  years  of  the  draft, 
military  personnel  were  viewed  as  essentially  cost-free  (which  was,  of  course,  an  illusion)  which 
resulted  in  little  attention  being  paid  to  personnel  issues  by  the  military.  However,  the  rapidly 
accelerating  shift  toward  technological  sophistication  in  the  weapon  and  support  systems  utilized 
by  the  services  has  led  to  significant  labor  demand  shifts  for  specific  types  of  labor.  It  has  also 
led  to  more  complex  career  behaviors  of  soldiers  themselves  (BDM,  1985).  The  MANPRINT 
program  itself  focuses  on  the  needs  and  capabilities  of  the  soldier  (Williges,  1990). 

MANPRINT  database  and  infonnation  sources  include  (Barila,  1987): 

o  Anny  Modernization  Information  Memorandum  (AMIM). 
o  Anny  Training  Requirements  and  Resources  System  (ATRRS). 
o  Basis  of  Issue  Plan  (BOIP)  /  Tables  of  Organization  and  Equipment  (TO&E). 
o  Defense  Technical  Information  Center  (DTIC). 
o  Manpower  Requirements  Criteria  (MARC)  Maintenance  Data  Base. 
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o  Personnel  Structure  and  Composition  system  (PERSACS). 
o  Qualitative  and  Quantitative  Personnel  Requirements  Information  (QQPRI). 
o  Standard  Study  Number  (SSN)  /  Line  Item  Number  (LIN), 
o  Center  for  Army  Lessons  Learned  (CALL), 
o  Directorate  of  Soldier  Advocacy  (DSA). 
o  Lorce  Structure  and  Programs  Directorate  (LSPD). 
o  Manning  Integration  directorate  (MID), 
o  Military  Occupational  Development  Directorate  (MODD). 
o  Personnel  Proponent  Coordination  directorate  (PPCD). 

Chapter  2  of  Army  Regulation  602-1  (Army,  1991),  states  that  Human  Lactors 
Engineering  (HLE)  is  defined  as  a  “comprehensive  technical  effort  to  integrate  into  Army 
doctrine,  materiel  development,  and  materiel  acquisition  (to  insure  operational  effectiveness)  all 
relevant  information  concerning”  (Weisz,  1989): 
o  Human  characteristics, 
o  Skill  capabilities, 
o  Performance, 
o  Anthropometric  data, 
o  Biomedical  factors, 
o  Safety  factors, 
o  Training. 

o  Manning  implications. 


The  MANPRINT  program  encourages  the  use  of  predecessor  or  reference  systems  in  the  analysis 
and  development  of  new  systems  (Lysaght,  Hill,  Dick,  Plamondon,  Linton,  Wierwille,  Zaklad, 
Bittner,  and  Wherry,  1989).  Army  Regulation  602-2  (Anny,  2001a)  identifies  the  critical 
MANPRINT  domains  as: 

•  Manpower.  The  number  of  personnel,  both  military  and  civilian,  required, 
authorized  and  potentially  available  to  train,  operate,  maintain,  and  support 
each  system  acquisition. 

•  Personnel.  The  human  aptitudes,  skills,  and  capabilities  required  to  operate, 
maintain,  and  support  a  system  in  peacetime  and  war. 

•  Training.  The  instruction  and  resources  required  to  provide  Army  personnel 
with  requisite  knowledge,  skills,  and  abilities  to  properly  operate,  maintain, 
and  support  Army  systems. 

•  Human  Lactors  Engineering.  The  comprehensive  integration  of  human 
capabilities  and  limitations  into  system  definition,  design,  development,  and 
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evaluation  to  promote  effective  soldier-machine  integration  for  optimal  total 
system  performance. 

•  System  Safety.  The  design  and  operational  characteristics  of  a  system  that 
minimize  the  possibilities  for  accidents  or  mishaps  caused  by  human  error  or 
system  failure. 

•  Health  Hazards.  The  systematic  application  of  biomedical  knowledge,  early  in 
the  acquisition  process,  to  identify,  assess,  and  minimize  health  hazards 
associated  with  the  systems  operation,  maintenance,  repair  or  storage,  such  as: 
Acoustic  energy,  toxic  substances  (biological  and  chemical),  oxygen 
deficiency,  radiation  energy,  shock,  temperature  extremes,  trauma  and 
vibration. 

•  Soldier  Survivability.  The  characteristics  of  a  system  that  reduce  fratricide  as 
well  as  reduce  detectability  of  the  soldier,  prevent  attack  if  detected,  prevent 
damage  if  attacked,  minimize  medical  injury  if  wounded  or  otherwise  injured, 
and  minimize  physical  and  mental  fatigue. 

Some  of  the  human  engineering  analysis  methods  used  to  conduct  MANPRINT 
assessments  include  (DOD,  1999): 

1  -  Mission  Analysis  is  used  to  define  what  tasks  the  total  system  (hardware,  software, 
and  liveware)  must  perform. 

2-  Task  description  /  analysis  is  a  method  designed  to  record  and  analyze  how  the 
human  is  involved  in  a  system  and  provides  an  organized  listing  of  how  the  human 
interacts  with  the  system. 

3-  Predetermined  time  standards  (PTSs),  which  are  internationally  recognized  time 
standards  used  for  work  measurement.  They  are  employed  to  estimate  performance 
times  for  tasks  that  can  be  decomposed  into  smaller  units  for  which  execution  times 
can  be  determined  or  estimated.  Examples  are  Methods  Time  Measurement  (MTM) 
and  Modular  Arrangement  of  Predetermined  Time  Standards  (MOD APTS). 

4-  Cognitive  Task  Analysis  (CTA)  is  a  task  analysis  method  that  focuses  on  describing 
the  cognitive  skills  and  abilities  needed  to  perform  a  task  proficiently  and  is  used  to 
analyze  and  understand  task  performance  in  complex  real  world  situations,  especially 
those  involving  change,  uncertainty,  and  time  pressure.  Example  is  the  Critical 
Decision  Method  (CDM),  which  emphasizes  the  elicitation  of  knowledge  from 
individuals  skilled  at  performing  a  given  task. 

5-  Functional  flow  diagrams  that  provide  a  detailed  outline  of  all  system  requirements. 

6-  Operational  Sequence  Diagrams  (OSD)  that  is  a  graphic  presentation  of  operator 
tasks  as  they  relate  sequentially  to  both  equipment  and  other  operators. 

7-  Flow  process  charts  (FPCs),  which  are  plots  of  the  sequence  of  operator  activities  or 
information,  transfer  as  a  part  of  a  system. 

8-  Decision  /  action  diagrams  which  show  the  flow  of  required  system  data  in  tenns  of 
operations  and  decisions. 
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9-  Action  /  infonnation  requirements  which  define  those  specific  actions  necessary  to 
perform  a  function  and,  in  turn,  those  specific  information  elements  that  must  be 
provided  to  perform  the  action. 

10-  Timeline,  which  is  a  plot  of  task,  flow  as  a  function  of  time. 

11-  Integrated  Computer  Aided  Manufacturing  Definition  (IDEF)  is  a  method  of  system 
modeling  that  enables  understanding  of  system  functions  and  their  relationships.  The 
IDEF  methodology  defines  a  system  in  terms  of  its  functions  and  its  inputs,  outputs, 
controls,  and  mechanisms  (ICOMS). 

+  IDEFO  models  the  functions  and  resources  of  a  system  rather  than  the 
process  flow  of  the  system. 

+  IDEF  IX  is  a  method  for  developing  logical  data  models  that  describe  the 
structure  of  information  within  a  system. 

12-  Function  allocation  trades  which  perform  preliminary  trade  off  studies  of  human 
machine  allocations  for  each  of  the  functions  being  considered. 

13-  Workload  analysis  that  provides  an  appraisal  of  the  extent  of  operator  or  crew  task 
loading,  based  on  the  sequential  accumulation  of  task  times. 

14-  Situation  Awareness  (SA)  analysis  where  SA  is  the  experience  of  fully 
understanding  what  is  going  on  in  a  given  situation,  of  seeing  each  element  within  the 
context  of  the  overall  mission  or  goal,  and  of  having  all  the  pieces  fit  together  into  a 
coherent  picture. 

15-  Link  Analysis  that  describe  the  interactions  between  components  in  a  system 
(human  or  machine). 

16-  Human  perfonnance  reliability  analysis  (HPRA),  which  is  an  analysis  of  the  factors 
that  detennine  how  reliably,  a  person  will  perform  within  a  system  or  process. 

Test  and  analysis  methods  used  to  conduct  MANPRINT  assessments  include  (DOD, 

1999): 

1-  Continuous  direct  observation. 

2-  Sampled  direct  observation. 

3-  Specification  compliance  summary  sheet. 

4-  Technical  manual  functional  evaluation. 

5-  Human  Factors  Engineering  Data  Guide  for  Evaluation  (HEDGE). 

6-  Use  of  Environment  and  engineering  measurement  equipment. 

7-  System  records  review. 

8-  Test  participant  history  record. 

9-  Interview. 

10-  Questionnaire. 
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11-  Motion  pictures. 

12-  Sound  recording. 

13-  Video  recording. 

14-  Still  photography. 

15-  Event  recording. 

16-  Secondary  task  monitoring. 

17-  Physiological  Instrumentation. 

1 8-  Physical  measurement. 

1 9-  Online  interactive  simulation. 

20-  Statistical  analysis. 

Unfortunately,  human  factors  continues  to  be  rediscovered  nearly  every  time  there  is  a 
well  publicized  disaster  in  which  “human  error”  is  involved  (Booher,  1990).  It  is  noted  that  it  is 
not  a  trivial  or  simple  matter  to  engineer  down  complexity  at  the  soldier-machine  interface 
(DePuy  and  Bonder,  1982).  The  MANPRINT  program  is  designed  to  enable  the  Anny  to 
overcome  many  of  these  oversights.  In  1985,  when  the  program  was  just  beginning,  it  was  noted 
that  many  times  a  system  program  manager  will  choose  not  to  implement  human  engineering 
changes  during  development  because  of  the  potential  for  cost  overruns  and  procurement  delays 
(O’Brien,  1985).  This  is  sometimes  still  true  today,  however,  the  MANPRINT  program  provides 
the  framework  for  the  development  of  systems  optimized  for  human  performance  especially  if 
the  human  performance  and  MANPRINT  assessments  are  made  early  enough  in  the  program 
development  cycle. 

2.1.7.  War. 

Some  thoughts  on  the  concept  of  war  itself  is  considered  necessary  as  a  basis  for  the 
study  of  techniques  to  deal  with  it  such  as  command  and  control.  A  recent  white  paper 
published  by  the  U.S.  Anny  stated  that,  at  its  most  fundamental  level,  war  is  a  brutal  contest  of 
wills  (Army,  2002b).  At  the  strategic  level  this  is  characterized  as  a  contest  of  national  will  and 
interests.  At  the  tactical  level  this  is  characterized  as  life  and  death  struggles  between  units  and 
individuals.  The  white  paper  goes  on  to  say  that  leaders  must  know  how  to  conduct  rapid  tactical 
decision  making  where  infonnation  superiority  via  a  web-  enhanced,  knowledge-  based  common 
operating  picture  is  the  key  to  force  effectiveness.  At  its  most  abstract  war  is  an  uncertain, 
mentally  complex,  physically  demanding,  and  intensely  emotional  experience.  Combat,  by  its 
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very  nature,  is  a  hostile  environment  intended  to  diminish  human  performance.  In  this  context,  it 
is  noted  that  the  sum  of  the  factors  which  degrade  human  perfonnance  may  be  much  greater  than 
the  sum  of  their  parts  (Parry,  Collins,  and  VanNostrand,  1990).  The  central  fact  of  combat  in 
war  is  danger  to  life  as  armies  exist  to  fight  and  fighting  means  casualties  (Williams,  1984). 

The  U.S.  Anny  has  defined  war  to  exist  at  three  levels.  These  are  strategic,  operational, 
and  tactical.  These  are  described  as  (Army,  2001c;  Gibbings,  1991): 

•  The  strategic  level  of  war  is  described  as  the  level  of  war  “at  which  a  nation  or 
group  of  nations  determines  national  or  alliance  security  objectives  and  develops 
and  uses  national  resources  to  accomplish  those  objectives.” 

•  The  operational  level  of  war  is  described  as  “the  level  of  war  at  which  campaigns 
and  major  operations  are  planned,  conducted,  and  sustained  to  accomplish 
strategic  objectives  within  theaters  or  areas  of  operations.” 

•  The  tactical  level  of  war  is  described  as  “the  level  of  war  at  which  battles  and 
engagements  are  planned  and  executed  to  accomplish  military  objectives  assigned 
to  tactical  units  or  task  forces.” 

General  Frederick  M.  Franks,  Jr.,  the  VII  Corps  Coimnander  during  Operation  Desert 
Storm  in  1 99 1 ,  is  quoted  as  saying  that  “modem  land  warfare  is  tough,  uncompromising,  and 
highly  lethal”  where  casualties  are  sudden,  and  the  combat  results  are  final  and  frozen  for  a 
lifetime  (Army,  2001c).  This  is  why  combat  leadership  is  so  demanding.  Future  military 
environments  will  be  evolutionary  and  chaotic,  often  presenting  ill-defined,  high-bandwidth 
information  dilemmas  that  proliferate  across  traditional  geopolitical  boundaries  under  highly 
stressful  conditions.  This  situation  is  termed  “cognition  in  the  wild”  (McNeese  and  Rentsch, 
2001). 

A  war  using  modem  weapons  systems  is  likely  to  be  both  intense  and  short.  There  are 
three  types  of  stressors  that  come  into  play  during  combat.  The  first  are  the  physical  stressors  of 
heat,  cold,  humidity,  noise,  overpressure,  toxic  substances  and  fatigue.  Then  there  are 
psychological  stressors  of  confinement  and  isolation,  crowding,  and  psychological  warfare. 
Finally,  there  are  NBC  stressors  from  the  effects  of  weapons  of  mass  destruction  such  as  nuclear, 
biological  and  chemical  weapons  (Kubala  and  Warnick,  1979). 

Few  activities  of  mankind  are  more  complex  than  combat  operations,  and  few  have  been 
studied  as  assiduously.  Notwithstanding  this,  man’s  understanding  of  the  process  of  warfare  is 
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incomplete  and  inadequate.  In  many  respects  certain  elements  known  to  have  an  important 
bearing  on  the  outcome  of  battle  can  neither  be  predetermined  nor  measured.  War  is  an  example 
of  the  complexity,  the  uncontrolled  variability,  and  the  impossibility  of  obtaining  and  recording 
desired  data  through  manipulation  and  observation  (Hausrath,  1971). 

2.2.  Issues  in  Human  Cognitive  Perfonnance. 

While  this  dissertation  is  focused  on  an  analysis  of  the  total  C2S,  it  is  realized  that  the 
human  operator  is  the  key  component  in  this  system.  For  this  reason,  an  understanding  of 
selected  concepts  of  human  performance,  especially  human  cognitive  perfonnance,  is  of 
paramount  importance  in  understanding  how  the  role  of  the  operator  affects  the  performance  of 
the  overall  system.  It  is  noted  that  each  topic  is  worthy  of  and  the  subject  of  much  research  in  its 
own  right,  and  are  presented  briefly  here  as  a  review  of  the  topic.  These  are  explored  briefly 
here  in  order  to  gain  some  understanding  of  how  each  might  be  considered  in  computer 
simulations  of  the  overall  system. 

2.2.1.  Expertise. 

Experts  see  the  world  differently.  They  see  things  the  rest  of  us  cannot.  One  view  of 
experts  is  that  they  have  accumulated  lots  of  knowledge.  Often  experts  do  not  realize  that  others 
are  unable  to  detect  what  seems  obvious  to  them  (Klein,  1998).  Some  of  the  many  things  experts 
can  see  that  are  invisible  to  everyone  else: 

•  Patterns. 

•  Anomalies. 

•  The  big  picture  (situation  awareness). 

•  The  way  things  work. 

•  Past  and  /  or  future  events  that  might  or  might  not  happen. 

•  Opportunities  and  improvisations. 

•  Differences  that  are  too  small  for  novices  to  detect. 

•  Their  own  limitations. 

People  with  greater  expertise  have  a  larger  resource  of  procedures  to  apply.  They  notice 
problems  more  quickly  and  have  a  have  richer  mental  simulation  to  use  in  diagnosing  problems 
and  in  evaluating  courses  of  action.  In  other  words,  they  have  more  analogies  to  draw  upon.  On 
the  negative  side,  expertise  can  cause  problems  by  causing  individuals  to  view  problems  in 
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stereotyped  ways.  Expert  systems  are  computer  programs  that  embody  domain-specific 
knowledge  and  that  perform  at  levels  typical  of  human  experts,  but  not  necessarily  in  exactly  the 
same  manner  as  human  experts  (Cooke,  1999). 

2.2.2.  Situation  Awareness. 

Situation  awareness  (SA)  is  defined  as  the  perception  of  the  elements  in  the  environment 
within  a  volume  of  time  and  space,  the  comprehension  of  their  meaning,  and  the  projection  of 
their  status  in  the  near  future  (Endsley,  1988).  It  is  a  state  of  knowledge  from  the  processes  used 
to  achieve  that  state  (Endsley,  1995b).  SA  refers  to  the  up-to-the-minute  cognizance  required  to 
operate  or  maintain  a  system  (Adams,  Tenney,  and  Pew,  1995).  One  view  of  SA  is  that  it  is  an 
adaptive,  externally  directed  consciousness  (Smith  and  Hancock,  1995).  It  can  be  described 
simply  as  “knowing  what  is  going  on  so  that  one  can  figure  out  what  to  do”  (AGARD,  1998)  or 
simply  knowing  what  is  going  on  around  you  (Endsley,  2000)  or  the  up  to  the  minute  cognizance 
or  awareness  required  to  move  about,  operate  equipment,  or  maintain  a  system  (Pew  and  Mavor, 
1998h).  Another  definition  states  that  SA  is  the  perception  of  reactions  to  a  set  of  changing 
events  (Klein,  2000a).  The  definition  of  SA  has  evolved  to  a  description  of  an  operator’s 
comprehension  of  a  complex  system  in  which  the  environment  is  dynamically  changing  and  in 
which  the  operator  is  responsible  for  maintaining  or  achieving  particular  states  or  goals  (Durso 
and  Gronlund,  1999).  Operator  SA  is  comprised  of  the  total  understanding  about  the  physical 
environment,  system  states,  one’s  own  status,  etc.  This  awareness,  or  knowledge,  forms  the 
basis  for  making  critical  decisions.  A  more  formal  definition  of  SA  is  that  it  is  “the  perception  of 
the  elements  in  the  environment  within  a  volume  of  time  and  space,  the  comprehension  of  their 
meaning  and  the  projection  of  their  status  in  the  near  future”.  SA  can  be  assessed  with  objective 
or  subjective  ratings  and  may  be  inferred  from  other  measures  of  perfonnance  (AGARD,  1998). 

Historical  evidence  has  shown  that  in  periods  of  rapid  movement  and  intensive  fighting 
the  “fog  of  battle”  becomes  unforgettably  evident  with  a  lack  of  information  not  only  in  the 
wider  context  but  also  of  what  was  happening  in  one’s  own  immediate  situation.  A  very  striking 
feature  of  combat  is  the  prevalence  of  extreme  restriction,  ignorance,  and  uncertainty.  The  chaos 
of  battle  obscures  nearly  everything  (Williams,  1984). 

The  enhancement  of  SA  has  become  a  major  design  goal  in  the  development  of  system 
interfaces  between  the  operator  and  the  hardware  and  software  of  the  system.  One  of  the  chief 
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reasons  to  measure  SA  is  for  the  purpose  of  evaluating  new  system  and  interface  designs.  Three 
levels  of  SA  are  defined  as  (Endsley,  2000): 

•  Level  1  SA  -  Perception  of  important  information. 

•  Level  2  SA  -  Comprehension  of  how  people  combine,  interpret,  store,  and  retain 
information. 

•  Level  3  SA  -  Projection  or  the  ability  to  forecast  future  situation  events  and 
dynamics.  This  marks  operators  who  have  the  highest  level  of  understanding  of 
the  situation  or  highest  SA. 

One  way  to  study  SA  is  through  critical  incidents,  which  can  be  presented  in  the  form  of 
simulations,  workplace  observations,  or  can  be  elicited  and  /  or  probed  through  interviews  that 
generate  narratives  from  memory.  The  acquisition  and  development  of  SA  by  a  system  operator 
is  an  active  process  of  guided  infonnation  seeking  rather  than  passively  receiving  and  storing 
details  (Klein,  2000a). 

Individual  situation  models  and  shared  mental  models  can  be  considered  precursor 
products  of  team  SA  in  that  they  are  cognitive  structures  that  are  brought  to  the  task  of 
interpreting  the  situation  by  team  members.  The  shared  mental  model  is  a  precursor  to  team  SA 
where  each  team  member  brings  to  the  task  not  only  momentary  or  transient  understandings  of 
the  current  situation,  i.e.,  individual  situation  models,  but  also  long  tenn,  relevant  individual 
knowledge  pertinent  to  the  environment,  task,  or  team  members.  The  information  that  is  critical 
for  SA  is  not  always  obvious,  even  to  an  expert  (Cooke,  Stout,  and  Salas,  2001). 

It  is  hypothesized  that  working  memory  constitutes  the  main  bottleneck  for  SA.  SA  is  a 
complex  process  of  perception  and  pattern  matching  greatly  limited  by  both  working  memory 
and  attention  capacities.  Several  mechanisms,  attention  sharing  and  automated  processing,  may 
serve  to  alleviate  these  limitations  to  some  degree.  Attention  can  also  be  seen  as  an  important 
constraint  on  SA.  Direct  attention  is  needed  for  not  only  perception  and  working  memory 
processing,  but  also  for  decision-making  and  response  executions  (Endsley,  1988). 

Some  general  principles  for  improving  SA  are  (Endsley,  1988): 

•  Information  presentation  should  feature  the  grouping  of  infonnation  in  terms  of 
spatial  proximity  and  the  use  of  multiple  information  imbedded  within  objects. 

•  The  use  of  information  grouping  should  tie  multiple  attributes  to  each  object 
while  minimizing  the  number  of  objects  presented. 

•  Information  presentation  should  allow  rapid  access  to  long-tenn  memory  storage. 
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•  Holistic  processing  strategies  should  be  employed. 

•  Information  filtering  by  the  system  should  be  employed  to  reduce  infonnation 
processing  filtering  requirements  on  the  operator. 

•  Information  that  allows  for  the  detennination  of  trend/rate  of  change  of 
components  should  be  available  to  the  operators. 

•  The  most  important  information  should  be  the  most  salient  perceptually  to  insure 
focused  attention. 

•  Peripheral  vision  can  be  employed  for  the  non-attentive  monitoring  of  qualitative 
state  or  changes  in  simple  secondary  information. 

•  Verbal  information  requirements  on  short-term  memory  should  be  minimized, 
particularly  for  spatial  information. 

•  Additional  types  of  infonnation  input,  such  as  auditory  or  tactile,  can  be  utilized 
to  provide  information  simultaneously  to  the  visual  channel. 

•  Methods  of  providing  simultaneous  access  to  secondary  information  should  be 
employed  which  will  not  induce  decrements  on  the  primary  task. 

•  Spatial  information  should  be  rapidly  relatable  to  the  operator’s  cognitive  map 
and  their  orientation  in  it. 

•  Designs/technologies  should  be  employed  which  will  reduce  pilot  workload  and 
improve  upon  the  quality  of  information  needed. 

The  impact  of  a  person  achieving  or  not  achieving  good  SA  can  be  significant.  In  general,  it  can 

be  expected  that  poor  performance  will  occur  when  SA  is  incomplete  or  inaccurate  (Endsley, 

1995b).  Current  military  thinking  insists  that  SA  provides  an  information  advantage  that 

translates  to  success  to  the  military  decision  maker  who  can  achieve  and  maintain  battlefield  SA 

(Lee,  1999).  Judging  the  level  of  SA  in  an  operator  when  faced  with  multiple,  competing 

demands  on  their  attention  during  system  operations  is  important  in  evaluating  the  effects  of 

attempts  to  improve  SA.  Global  measures  of  SA  across  many  elements  of  interest  is  desirable  if 

designers  are  to  be  able  to  evaluate  the  impact  of  projected  design  concepts  on  operator  SA 

(Endsley,  1995a).  These  concepts  are  not  new.  Over  2400  years  ago  the  Chinese  military  genius 

Sun  Tzu  wrote:  “If  you  know  the  enemy  and  know  yourself,  you  need  not  fear  the  result  of  a 

hundred  battles  (Tzu,  1910).” 

2.2.3.  Stress. 

Stress  is  an  important  concept  in  the  context  of  this  research  according  to  its  potential  to 
affect  individual  and  group  performance  and  decision-making.  The  issue  of  stress  and 
performance  is  of  special  importance  on  the  high  tech  battlefield  because  of  the  possibility  that 
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equipment  may  not  be  optimally  operated  by  soldiers  whose  cognitive  and  sensory-motor  skills 
are  degraded  by  stress  (Buckalew,  1990).  While  the  evidence  that  shows  that  stress  results  in 
decision  making  errors  is  not  convincing,  stress  does  effect  the  way  decisions  are  made, 
however,  it  does  not  cause  bad  decisions  to  be  made  that  are  based  on  available  information. 
Stress,  in  itself,  does  not  result  in  faulty  decision-making  but  it  may  limit  the  information  that  is 
considered  while  making  the  decisions.  Stressors  such  as  time  pressure,  noise,  and  ambiguity 
can  result  in  effects  such  as  less  information  being  gathered  to  support  decision  making,  a 
disruption  of  working  memory  to  sort  out  details,  and  a  distraction  to  the  task  at  hand.  The 
evidence  that  shows  that  stress  results  in  decision  errors  is  not  convincing  nor  does  stress  effect 
the  way  decisions  are  made  as  it  does  not  cause  bad  decisions  based  on  the  available  information. 
Stressors  may  result  in  the  following  effects  (Klein,  1998): 

•  Not  as  much  information  can  be  gathered. 

•  The  ability  to  use  working  memory  to  sort  things  out  is  disrupted. 

•  The  attention  to  the  task  at  hand  is  distracted. 

Some  key  points  in  decision  making  are  (Klein,  1998): 

•  Decision  biases  do  not  seem  to  explain  poor  decisions. 

•  Stress  does  not  result  in  faulty  decision  making  strategies  but  may  limit  the 
information  to  be  considered  in  making  the  decisions. 

•  Most  poor  decisions  may  result  from  having  inadequate  knowledge  and  expertise. 

•  Experience  does  not  translate  directly  into  expertise  if  the  domain  is  dynamic, 
feedback  is  inadequate,  and  the  number  and  variety  of  experiences  is  too  small. 

The  types  of  stressors  include  extreme  temperature,  noise,  sleep  deprivation,  time 
constraints,  frustration,  and  performance  pressure,  etc.  Stress  affects  the  speed/accuracy  tradeoff 
and  the  effect  of  stress  on  decision  making  can  largely  be  explained  in  the  terms  of  less  time 
being  allotted  to  sub  tasks  such  as  gathering  information.  Each  stressor  requires  attention  for 
adjusting  to  and  managing  the  stress  itself  and  this  may  be  seen  as  constituting  a  secondary  task, 
so  that  degraded  performance  under  stress  may  also  be  a  function  of  fewer  cognitive  resources 
available  (Klein  and  Crandall,  1996).  Stress  casualties  can  be  expected  to  be  as  high  as  1:4  to 
1 :3  in  comparison  to  casualties  such  as  wounded  in  action  (WIA).  Temperature  extremes,  rough 
terrain,  and  high  altitudes  cause  fatigue  that  degrades  performance  (Vandivier,  1990). 

In  the  case  of  sleep  deprivation,  soldiers  become  militarily  ineffective  after  only  48  to  72 
hours  with  no  sleep  and  suffer  a  degradation  of  75  percent  in  performance  on  most  tasks  after  72 
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hours  of  work  with  no  sleep.  In  addition,  cognitive  abilities  begin  to  degrade  as  soon  as  18  hours 
into  sustained  operations.  For  physical  performance  and  routine  motor  tasks,  the  degradation  is 
less  rapid  than  cognitive  tasks.  High  levels  of  cohesiveness,  morale,  and  motivation  can  reduce 
stress  casualties.  Assuming  all  other  factors  are  equal,  leadership,  training,  and  high  mental 
aptitude  can  result  in  better  performance  on  the  battlefield  (Vandivier,  1990). 

2.2.4.  Decision  Making. 

A  decision-making  episode  occurs  whenever  the  flow  of  action  is  interrupted  by  a  choice 
between  conflicting  alternatives.  A  decision  is  made  when  one  of  the  competing  alternatives  is 
executed,  producing  a  change  in  the  environment  and  yielding  consequences  relevant  to  the 
decision  maker  (Pew  and  Mavor,  1998b).  The  crucial  activities  for  decision  making  are  actions 
whose  controlled  execution  consolidates  fragments  of  policy  that  are  lying  around,  gives  them 
direction,  and  closes  off  other  possible  arrangements.  Decisions  that  are  tied  more  closely  to 
action  are  more  likely  to  contain  improvisation  (Weick,  1983).  Also,  planning,  or  the  generation 
of  a  plan,  is  critical  to  successful  operations  where  it  plays  a  key  role  in  the  tactical  decision¬ 
making  process  in  the  military  throughout  all  echelons  (Pew  and  Mavor,  1998g). 

Some  comments  (Klein,  1998)  on  inaccurate  decision  making  include  the  statement  that 
decisions  can  be  considered  to  be  poor  if  the  knowledge  gained  would  lead  to  a  different  decision 
if  a  similar  situation  arose.  Poor  decisions  can  also  be  caused  by  factors  such  as  lack  of 
experience  according  to  naturalistic  decision-making  theory.  Some  of  the  causes  of  poor 
decision-making  are: 

•  Lack  of  experience. 

•  Lack  of  infonnation. 

•  Mental  simulation,  the  de  minimus  error  where  the  signs  of  the  problem  were 
noticed  but  were  explained  away. 

In  complex  and  dynamic  environments,  attention  demands  resulting  from  infonnation  overload, 
complex  decision  making,  and  multiple  tasks  can  quickly  exceed  a  person’s  attention  capacity 
(Endsley,  1995b). 

The  evidence  that  shows  that  stress  results  in  decision  errors  is  not  convincing  nor  does 
stress  effect  the  way  decisions  are  made  but  it  does  not  cause  bad  decisions  based  on  the 
available  information.  Stressors  such  as  time  pressure,  noise,  and  ambiguity  result  in  the 
following  effects  (Klein,  1998): 
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•  Not  as  much  information  can  be  gathered. 

•  The  ability  to  use  working  memory  to  sort  things  out  is  disrupted. 

•  The  attention  to  the  task  at  hand  is  distracted. 

Some  key  points  in  decision  making  are  (Klein,  1998): 

•  Decision  biases  do  not  seem  to  explain  poor  decisions. 

•  Stress  does  not  result  in  faulty  decision  making  strategies  but  may  limit  the 
information  to  be  considered  in  making  the  decisions. 

•  Most  poor  decisions  may  result  from  having  inadequate  knowledge  and  expertise. 

•  Experience  does  not  translate  directly  into  expertise  if  the  domain  is  dynamic, 
feedback  is  inadequate,  and  the  number  and  variety  of  experiences  is  too  small. 

The  field  of  naturalistic  decision  making  (NDM)  is  a  recent  approach  to  describing  how 
system  operators  actually  make  judgments  and  decisions,  during  emergencies  as  well  as  routine 
conditions  (Klein,  1993).  NDM  is  concerned  with  poorly  defined  procedures  as  opposed  to 
laboratory  studies  where  decision-making  is  distinct  from  problem  solving.  Skilled  military 
commanders,  for  example,  will  evaluate  a  plan  of  action  by  mental  simulation  and  will  apply  a 
sense  of  predictability  to  notice  that  their  adversary  can  anticipate  their  moves  and  will  take  the 
necessary  precautions  to  prevent  it  (Klein,  1998). 

Recognition  primed  decision  (RPD)  making  is  the  process  of  making  decisions  based  on 
experiential  knowledge.  For  example,  an  experienced  commander’s  secret  was  that  their 
experience  let  them  see  a  situation,  even  a  nonroutine  one,  as  an  example  of  a  prototype,  so  they 
knew  the  typical  course  of  action  right  away  even  though  it  was  not  easy  to  classify  the  decision 
points.  Experts  are  able  to  see  the  world  differently  and  have  the  ability  to  recognize  things  that 
others  cannot.  One  view  of  experts  is  that  they  have  accumulated  lots  of  knowledge  (Klein, 
1998). 

The  RPD  model  of  decision-making  describes  a  decision  strategy  commonly  employed 
by  proficient  personnel  called  upon  to  make  decisions  in  operational  settings  by  high  risk,  time 
constraints,  and  ambiguous  or  incomplete  information.  It  is  a  decision  strategy  that  appears  well 
suited  for  operational  settings  marked  by  time  pressure,  ambiguity,  incomplete  information,  and 
ill  defined  and  shifting  goals.  A  key  question  from  the  NDM  perspective  is  how  people  are  able 
to  use  experience  to  handle  difficult  conditions.  The  RPD  model  asserts  that  experience  enables 
decision  makers  to  recognize  the  essential  characteristics  of  a  situation,  and  thereby  to  identify 
feasible  goals  and  plausible  courses  of  action  through  a  fusing  of  two  processes,  situation 
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assessment  and  mental  simulation.  The  hypothesis  for  the  RPD  model  claims  that  decision 
makers  quickly  recognize  a  favorite  option,  and  a  next  best  option,  and  that  the  decision  making 
consists  of  trying  to  show  that  the  favorite  option  dominates  the  next  best  option  on  all 
evaluation  dimensions  (Klein  and  Crandall,  1996). 

Findings  and  conclusions  of  the  RPD  model: 

•  Experienced  decision  makers  rely  more  on  situation  assessment  while  novices 
rely  more  on  option  evaluation  strategies. 

•  Situation  assessment  seems  to  involve  schematic  or  prototypical  knowledge  of 
cues,  goals,  and  expectancies  that  apply  to  a  given  class  of  events. 

•  Whereas  experts  and  novices  notice  the  same  cues  in  a  situation,  novices  draw 
fewer  inferences  based  on  these  cues  and  tend  to  miss  the  tactical  implications  of 
the  situational  cues. 

•  In  the  command  and  control  domain  decisions  are  most  likely  to  be  made  without 
any  conscious  deliberation  between  option  alternatives. 

•  When  deliberation  does  occur,  decision  makers  are  more  likely  to  use  serial 
evaluation  strategies  than  concurrent  evaluation  of  options.  Serial  strategies 
appear  to  offer  a  means  of  minimizing  the  calculation  burden  as  well  as 
maximizing  the  speed  with  which  a  decision  may  be  implemented. 

•  Serial  evaluation  is  associated  with  satisficing  rather  than  optimizing  strategies, 
and  is  preferred  under  time -pressured  conditions. 

•  Options  are  frequently  evaluated  through  the  use  of  images  or  a  “mental  model” 
that  operates  as  a  simulation  forjudging  whether  an  option  will  be  successful  in  a 
specific  case. 

•  Expert  decision  makers  rely  on  a  process  of  “progressive  deepening”  or  reasoning 
into  the  future. 

•  Analogical  reasoning  is  infrequently  reported,  suggesting  that  the  processes 
involved  in  selecting  and  using  analogues  are  relatively  automatic  and 
unconscious. 

•  When  analogues  are  used  (often  to  address  non-routine  aspects  of  a  problem), 
they  are  critical  to  option  selection.  Thus,  inappropriate  analogues  are  a  primary 
cause  of  errors. 

•  Time  pressure  does  not  affect  the  quality  of  decisions  made  by  experts  as  much  as 
novices,  due  to  experts  greater  reliance  on  rapid  recognition  processes. 

The  RPD  model  shows  how  the  proficient  decision  maker  becomes  aware  of  events  that 
have  occurred,  and  relies  on  experience  to  recognize  the  events  as  largely  typical  (Klein  and 
Crandall,  1996). 
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The  U.S.  Army  approach  to  decision  making  is  called  the  Military  Decision  Making 
Process  (MDMP).  This  process  was  first  described  in  the  1932  version  of  FM  101-5,  Staff 
Officers  Field  Manual,  Part  I,  as  the  “Estimate  of  the  Situation”  (Army,  1997).  The  steps  in  that 
process  included  a  statement  of  the  mission,  the  disposition  of  the  enemy  forces,  an  analysis  of 
the  enemy  situation,  an  analysis  of  the  friendly  force  situation,  and  the  decision  to  be  made 
regarding  the  next  action  (Charlton,  1997).  Today’s  MDMP  is  diagrammed  in  Figure  10  and 
consists  of  six  steps  which  are  receipt  of  mission,  mission  analysis,  course  of  action  (COA) 
development,  COA  analysis,  COA  comparison,  and  COA  approval  and  orders  production. 


NOTE1:  Commander  may 
conduct  phases  independently 
or  in  conjunction  with  staff. 
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NOTE  4:  At  any  time  during 
execution  and  assessment, 
situation  may  require  the 
process  to  start  again. 


Figure  10  -  The  Military  Decision  Making  Process 
(Army,  1997) 


However,  the  fundamental  dilemmas  faced  by  the  Army  using  the  MDMP  are  fivefold  (Centric 
and  Salter,  1999): 
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•  While  the  Army  has  become  increasingly  involved  since  the  demise  of  the  cold 
war  in  the  operational  level  of  war  (for  example,  peacekeeping  and  direct  action 
as  opposed  to  full  scale  conventional  warfare)  there  is  currently  little  experience 
at  this  level. 

•  While  the  MDMP  describes  a  tactical  process,  it  is  also  being  used  at  the  joint 
planning  level  because  there  is  no  alternative. 

•  The  MDMP  was  designed  for  the  Army’s  mission  during  the  Cold  War  as 
opposed  to  current  missions  that  are  focused  on  the  operational  level. 

•  While  the  MDMP  works  well  in  potentially  time  consuming  and  complex  and 
uncertain  environments  typical  of  full-scale  land  warfare,  it  has  deficiencies  in 
missions  that  can  call  for  time  compressed  decision  cycles. 

•  The  MDMP  does  not  accurately  model  the  way  experienced  decision  makers 
naturally  make  decisions  (reference,  for  example,  Klein’s  RPD  model  (Klein, 
1998)). 

There  is  a  considerable  discussion  in  the  literature  concerning  ways  to  optimize,  enhance, 
and/or  generally  improve  the  ways  that  decisions  are  made  especially  in  the  military  under 
combat  conditions.  It  is  noted  that  properly  designed  decision  support  systems  should  include 
provisions  for  the  heuristics  that  are  likely  to  be  employed  by  a  decision  maker  when  faced  with 
rapidly  changing  and  infonnation  intensive  situations  along  with  incomplete  or  questionable  data 
(Colton  and  Ganze,  1993). 

2.2.5.  Task  Performance. 

The  ability  of  individuals  and  teams  to  perform  tasks  as  part  of  the  overall  function  of 
battlespace  management  is  very  important  to  the  successful  operation  of  a  C2S.  Human  task 
performance  has  been  described  in  a  variety  of  models  and  descriptions  and  a  widely  accepted 
description  is  in  the  multiple  resources  model.  The  multiple  resources  model  presents  a  fonnal 
framework  for  describing  the  extent  to  which  two  human  infonnation  channels,  each  described 
by  one  of  four  formats  of  perception,  two  codes  of  central  processing  and  response,  and  three 
stages  of  processing,  will  compete  or  interfere  with  each  other. 

Anatomical  research,  looking  at  the  different  human  cognitive  processing  abilities,  has 
established  that  visual  information  presented  to  the  right  visual  field  is  relayed  directly  to  the  left 
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cerebral  hemisphere,  and  left  visual  field  information  directly  accesses  the  right  hemisphere. 
Since  the  left  and  right  hemispheres  are  generally  associated  with  verbal  and  spatial  processing 
respectively,  the  direct  access  principle  asserts  that  infonnation  of  a  given  code  (verbal  or 
spatial)  will  be  best  processed  if  it  is  presented  to  the  fovea  and  contra-lateral  visual  field  ( i.e., 
verbal,  right  field;  spatial,  left  field).  One  of  the  strong  intuitive  appeals  of  the  use  of  voice  and 
auditory  displays  is  that  information  thus  displayed  may  be  perceived  without  disrupting  visual 
processing. 

The  multiple  resources  model  predicts  that  as  tasks  occupy  different  “cells”  of  processing 
resources,  interference  between  them  will  be  reduced.  In  contrast  to  the  weak  effects  observed 
when  factors  such  as  compatibility  and  competition  are  in  opposition,  the  magnitude  of  effects 
when  the  two  factors  work  together  can  be  quite  pronounced.  Changing  a  task  from  an 
incompatible  resource  competitive  configuration  to  a  compatible  configuration  with  separate 
resources  has  three  positive  influences.  First,  dual  task  performance  decrement  is  reduced  when 
separate  resources  are  employed.  Second,  compatibility  produces  a  higher  baseline  of 
performance.  Third,  high  compatibility  also  reduces  the  resource  demands  of  a  task,  and  so 
further  reduces  the  level  of  dual  task  interference. 

For  example,  if  the  choice  is  to  be  made  as  to  whether  to  use  auditory  displays  for 
presenting  information  for  a  spatial  or  verbal  task,  the  greatest  gains  by  far  will  result  from 
allocating  the  display  to  the  verbal  task.  Any  advantages  of  using  an  auditory  display  for  the 
spatial  task  will  be  offset  and  nullified  by  the  low  level  of  compatibility.  The  greatest  gains  to 
auditory  displays  will  be  realized  by  the  verbal  tasks.  Examples  of  visual  and  auditory  verbal 
and  spatial  displays  are: 

•  Visual  spatial  fonnat  can  appear  as  an  arrow  emanating  from  the  symbol  and 
pointing  up  or  down. 

•  Visual  verbal  format  can  use  the  word  “up”  or  “down”  that  appears  beside  the 
corresponding  symbol. 

•  Auditory  verbal  format  can  consist  of  the  symbol  flashing  concurrently  with  the 
spoken  word  “up”  or  “down”. 

•  Auditory  spatial  format  can  consist  of  a  high  or  low  tone  presented  in  conjunction 
with  the  flashing  symbol. 
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Other  considerations  include  the  observation  that  the  auditory  modality  may  present 
problems  in  noisy  environments,  while  the  visual  modality  is  more  sensitive  to  disruptive  effects 
of  anoxia,  high  G  forces,  and  vibration  (Wickens,  1984). 

2.2.6.  Team  Performance. 

A  team  is  defined  as  a  distinguishable  set  of  two  or  more  people  who  interact, 
dynamically,  interdependently,  and  adaptively  toward  a  common  and  valued  goal  /  objective  / 
mission,  who  have  each  been  assigned  specific  roles  or  functions  to  perform,  and  who  have  a 
limited  life-span  of  membership  (Mathieu  et  al.,  2000;  Salas,  Dickinson,  Converse,  and 
Tannenbaum,  1992).  A  similar  definition  states  that  a  team  is  defined  as  two  or  more 
interdependent  individuals  perfonning  coordinated  tasks  toward  the  achievement  of  specific  task 
goals  who  interact,  dynamically,  interdependently,  and  adaptively  toward  a  common  and  valued 
goal/objective/mission.  They  have  each  been  assigned  specific  roles  or  functions  to  perform,  and 
have  a  limited  life  span  of  membership  (Salas  et  al.,  1992). 

Team  performance  has  two  primary  components.  These  are  individual  task  behaviors 
and  coordinated  task  related  processes/functions/behaviors  (Fleishman  and  Zaccaro,  1992). 

Work  teams  of  individuals  form  a  subconscious  identity  that  can  be  described  as  the  team  mind. 
Some  of  the  functions  of  the  team  mind  include  the  ability  to  hold  information  for  brief  periods 
of  time  (working  memory),  the  ability  to  store  information  permanently  (long  term  memory),  the 
ability  to  focus  on  only  one  thing  at  a  time  (limited  attention),  sensory  mechanisms  that  convert 
mechanical  energy  into  patterns  of  neural  activity  (perceptual  filters),  and  the  ability  to  acquire 
new  procedures  while  discarding  inefficient  behaviors  and  determining  how  to  become  more 
effective  (Klein,  1998).  The  nature  and  type  of  problems  experienced  by  teams  in  operational 
settings  are  naturalistic  and  involve  many  elements  that  may  emerge  in  situations  (McNeese  and 
Rentsch,  2001).  Most  decision-making  activities  in  large  scale  systems  and  organizations  are 
performed  by  a  team  of  people  who  are  interconnected  by  communication  networks  (Kleinman, 
Luh,  Pattipati,  and  Serfaty,  1992). 

Key  points  in  describing  team  performance  include  the  recognition  that  a  team  is  an 
intelligent  entity.  Its  cognition  can  be  inferred  from  such  things  as  the  team  behaviour,  the 
contents  of  the  team’s  collective  consciousness,  and  the  team’s  preconscious.  The  collective 
team  mind  has  the  ability  to  develop  basic  competencies  and  routines.  It  forms  a  clear  identity 
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while  learning  to  manage  the  flow  of  ideas  and  learning  to  monitor  itself  to  adjust  its  thinking 
when  necessary  (Klein,  1998). 

The  ability  of  the  individuals  to  effectively  function  together  as  a  team  is  dependent  on 
many  factors  and  is  vitally  important  to  the  success  of  many  organizations  and  work  groups. 

One  of  the  factors  that  has  been  studied  is  the  differential  effects  from  human  interaction  face  to 
face  versus  through  remote  communications  systems.  Systems  in  which  visual  cues  such  as 
selective  gaze  are  absent  produce  no  differences  in  turn  taking  or  in  any  other  aspect  of  the 
structure  of  conversation.  In  fact,  (in  this  experiment)  turn  taking  was  unaffected  even  when 
visual  information  was  completely  absent.  However,  in  videoconferencing,  failure  to  make  eye 
contact  tends  to  be  a  problem  because  of  the  separation  of  the  camera  and  monitor.  One 
implication  of  multiparty  videoconferences  involving  several  sites  using  the  Picture  In  Picture 
(PIP  or  Hollywood  Squares)  approach  is  that  it  fails  to  support  selective  gaze  and  selective 
listening  (Sellen,  1995). 

Some  of  the  most  profound  changes  in  social  and  organizational  behavior  in  this  century 
can  be  traced  to  tools  that  support  remote  individual  and  team  performance  cooperation.  While 
collaborative  work  at  a  distance  will  be  difficult  to  do  for  a  long  time,  if  not  forever,  effective 
communication  between  people  requires  that  the  communicative  exchange  take  place  with 
respect  to  some  level  of  common  ground.  Common  ground  refers  to  that  knowledge  that  the 
participants  have  in  common,  and  they  are  aware  that  they  have  it  in  common  (Olson  and  Olson, 
2000).  Common  ground  means  two  or  more  individuals  working  together  and  articulating 
perspectives  on  problems  to  jointly  arrive  at  a  shared  meaning  that  references  goals,  tenns, 
content,  context,  and  process  of  a  situation  (McNeese  and  Rentsch,  2001).  For  example,  people 
describe  the  same  event  or  idea  quite  differently  talking  to  a  spouse,  a  coworker,  a  distant 
relative,  a  neighbor,  a  stranger  from  across  the  country  and  a  stranger  from  overseas  (Olson  and 
Olson,  2000).  In  1898  Arthur  Mee  stated: 

“  If,  as  it  is  said  to  be  not  unlikely  in  the  near  future,  the  principle  of  sight  is  applied  to 
the  telephone  as  well  as  that  of  sound,  earth  will  be  in  truth  a  paradise,  and  distance  will  lose  its 
enchantment  by  being  abolished  altogether  (Mee,  2000)” 

The  shared  mental  model  theory  offers  an  explanation  of  what  the  mechanisms  of 
adaptability  might  be,  that  is,  how  teams  can  quickly  and  efficiently  adjust  their  strategy  “on  the 
fly.”  The  function  of  shared  mental  models  is  to  allow  team  members  to  draw  on  their  own  well- 
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structured  knowledge  as  a  basis  for  selecting  actions  that  are  consistent  and  coordinated  with 
those  of  their  teammates.  A  person’s  mental  model  of  the  domain  is  described  as  the  structure  in 
which  the  declarative  knowledge  framework  is  organized.  The  shared  mental  models  approach 
proposes  that  the  overlap  of  individuals’  mental  models  leads  to  greater  shared  expectations  and 
explanations  within  a  team  which  leads  to  improved  coordination,  communication  and  other 
team  behaviors  which  in  turn  leads  to  superior  team  perfonnance  (Banks  and  Millward,  2000; 
Zachary,  Ryder,  and  Hicinbothom,  1998).  Military  organizations  use  teams  to  plan,  initiate,  and 
coordinate  battles  (Streufert  and  Nogami,  1992).  These  teams  that  are  operating  complex 
command  and  control  networks  with  life  and  death  literally  hanging  in  the  balance,  are  among 
the  most  highly  skilled  and  trained  teams  in  existence.  Nevertheless,  history  has  shown  that 
breakdowns  do  occur  and  even  these  teams  make  mistakes. 

All  models  of  team  functioning  share  an  input-process-outcome  (I-P-O)  framework 
according  to  the  general  systems  theory.  Inputs  to  such  models  are  conditions  that  exist  prior  to 
a  performance  episode  and  may  include  member,  team,  and  organizational  characteristics. 
Processes  describe  how  team  inputs  are  transformed  into  outputs.  Outcomes  are  results  and  by 
products  of  team  activity  that  are  valued  by  one  or  more  constituencies.  Types  of  outcomes 
include  performance,  i.e.,  quality  and  quantity,  team  longevity  and  team  members’s  affective 
reactions  (Mathieu  et  ah,  2000).  Examples  of  models  of  team  perfonnance  are  the  nonnative 
model  where  the  basic  assumption  is  that  organizational  context  and  group  design  (i.e.,  input 
variables)  affect  the  member  interaction  process,  and  that,  in  turn,  affects  the  quality  of  team 
performance  (i.e.,  the  output  variables).  The  time  and  transition  model  (TTM)  where  the  team  is 
assumed  to  start  out  with  a  method  of  performing  at  the  beginning  of  the  project  only  to  pursue  a 
different  strategy  halfway  through  the  task.  The  task  group  effectiveness  model  (TGFM)  where 
group  process  and  group  effectiveness  are  central  to  the  model  with  group  task  demands  serving 
as  a  moderating  factor.  The  team  evolution  and  maturation  model  (TEAM)  is  a  model  of  team 
performance  that  predicts  the  stages  that  teams  go  through  before,  during,  and  after  performance 
of  a  task.  The  team  performance  model  (TEM)  is  where  the  team  has  the  two  major  components 
of  task  behaviors  by  individuals,  and  task  functions  at  the  team  level.  Finally,  the  task  oriented 
model  (TOM)  emphasizes  that  team  perfonnance  is  a  function  of  the  subtasks  that  members  must 
perform  effectively  for  the  accomplishment  of  team  goals  (Salas  et  ah,  1992). 

A  taxonomy  of  team  perfonnance  is  presented  as  (Fleishman  and  Zaccaro,  1992): 
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I.  Orientation  Functions. 

A.  Infonnation  Exchange  Regarding  Member  Resources  and  Constraints. 

B.  Infonnation  exchange  Regarding  Team  Task  and  Goals  /  Mission. 

C.  Information  Exchange  Regarding  Environmental  Characteristics  and 
Constraints. 

D.  Priority  Assignment  Among  Tasks. 

II.  Resource  Distribution  functions. 

A.  Matching  Member  Resources  to  Task  Requirements. 

B.  Load  Balancing. 

III.  Timing  Functions  (Activity  Pacing). 

A.  General  Activity  Pacing. 

B.  Individually  Oriented  Activity  Pacing. 

IV.  Response  Coordination  functions. 

A.  Response  Sequencing. 

B.  Time  and  Position  Coordination  of  Responses. 

V.  Motivational  Functions. 

A.  Development  of  Team  Performance  Norms. 

B.  Generating  Acceptance  of  Team  Perfonnance  Norms. 

C.  Establishing  Team  Level  Performance  Rewards  Linkages. 

D.  Reinforcement  of  task  Orientation. 

E.  Balancing  Team  Orientation  with  Individual  Competition. 

F.  Resolution  of  Performance-Relevant  Conflicts. 

VI.  Systems  Monitoring  Functions. 

A.  General  Activity  Monitoring. 

B.  Individual  Activity  Monitoring. 

C.  Adjustment  of  Team  and  Member  Activities  in  Response  to  Errors  and 
Omissions. 

VII.  Procedure  Maintenance. 

A.  Monitoring  of  General  Procedural  Based  Activities. 

B.  Monitoring  of  Individual  Procedural  Based  Activities. 

C.  Adjustments  of  Nonstandard  Activities. 
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Another  view  of  a  team  performance  taxonomy  is  presented  as  (Nieva,  Fleishman,  and 
Rieck,  1985): 

I.  Team  Orientation  Functions 

A.  Elicitation  and  distribution  of  infonnation  about  team  goals. 

B.  Elicitation  and  distribution  of  information  about  team  tasks. 

C.  Elicitation  and  distribution  of  information  about  member  resources  and 
constraints. 

II.  Team  Organizational  Functions. 

A.  Matching  member  resources  to  task  requirements. 

B.  Response  coordination  and  sequencing  of  activities. 

C.  Activity  pacing. 

D.  Priority  assignment  among  tasks. 

E.  Load  balancing  of  tasks  by  members. 

III.  Team  Adaptation  Functions. 

A.  Mutual  critical  evaluation  and  correction  of  error. 

B.  Mutual  compensatory  performance. 

C.  Mutual  compensatory  timing. 

IV.  Team  Motivational  functions. 

A.  Development  of  team  performance  nonns. 

B.  Generating  acceptance  of  team  perfonnance  norms. 

C.  Establishing  team  level  perfonnance  -  rewards  linkages. 

D.  Reinforcement  of  task  orientation. 

E.  Balancing  team  orientation  with  individual  competition. 

F.  Resolution  of  performance  -  relevant  conflicts. 

2.2.7.  Workload. 

It  was  stated  20  years  ago  that  there  is  no  consistent  definition  of  mental  workload,  no 
agreement  on  how  to  measure  it,  and  no  single  universal  metric  describing  it  (Lysaght  et  al., 
1989;  Moray,  1982;  Williges  and  Wierwille,  1979).  This  statement  is  apparently  still  true  today. 
The  problem  is  that  it  is  not  known  for  certain  what  it  is  that  a  human  senses  when  making 
judgments  of  difficulty  as  there  is  no  mental  force  that  has  been  operationally  defined  with 
consistent  internal  dimensions.  Human  task  based  activities  during  system  performance  have 
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been  described  to  exist  in  the  four  categories  of  perceptual,  cognitive,  communications,  and 
motor  (Wierwille  and  Casali,  1983).  Mental  workload  clearly  relates  to  such  factors  as  operator 
stress  and  effort,  however,  these  factors  are  as  lacking  in  operational  definitions  as  mental 
workload.  The  fact  that  mental  workload  is  multidimensional  in  nature  presupposes  that  any  one 
measure  can  describe  its  effect.  Therefore,  attempts  to  measure  mental  workload  should  include 
multiple  measures  such  as  subjective  opinion,  spare  mental  capacity,  and  primary  task  measures 
along  with  physiological  indices  (Williges  and  Wierwille,  1979).  In  comparison  to  other 
scientific  inquiry,  there  has  not  been  a  lot  of  research  into  what  makes  a  human  experience 
excessive  mental  workload  (Moray,  1982). 

There  are  many  different  definitions  of  workload  that  arise  from  individual  situations  and 
contexts.  The  dictionary  (Webster,  1979)  defines  it  as:  “  the  amount  of  work  or  of  working  time 
expected  from  or  assigned  to  an  employee.”  However,  in  tenns  of  human  cognitive  perfonnance 
it  is  much  more  than  this  and  much  less  capable  of  being  precisely  defined.  One  definition  is  the 
capacity  to  perform  (Lysaght  et  al.,  1989).  It  can  also  be  described  as  the  demand  placed  upon 
humans  during  task  performance  (Rouse,  Edwards,  and  Hammer,  1993).  It  is  the  total  attention 
demand  placed  on  the  operators  as  they  perform  the  mission  tasks  (Hamilton  et  al.,  1991).  A 
more  formal  definition  is  related  to  subjective  opinion  of  the  operator  on  how  much  work  is 
being  performed.  Thus,  subjective  workload  is  that  load  that  is  perceived  by  the  operator  and 
may  fluctuate  as  a  function  of  experience,  sensory  acuity,  cognitive  flexibility,  affective 
condition,  and  state  of  fatigue  of  the  operator  (Warren,  Stern,  Eddy,  Horst,  Kramer, 

Parasuraman,  Sanquist,  and  Wilson,  1985). 

The  Army  defines  workload  as  the  amount  of  work,  stated  in  predetennined  work  units, 
that  organizations  or  individuals  perfonn  or  are  responsible  for  performing  (Army,  1983). 
Attention,  with  its  components  of  cognitive,  psychomotor,  and  sensory,  plays  a  major  role  in 
invoking  workload.  Workload  can  be  characterized  as  the  demand  on  each  of  these  components 
imposed  by  all  the  tasks  an  operator  is  currently  performing.  When  these  demands  exceed  the 
capacity  of  the  operator  then  the  condition  of  operator  overload  can  be  experienced.  Operator 
overload  is  defined  as  the  level  of  workload  at  which  operator  performance  begins  to  degrade 
(Hamilton  et  al.,  1991).  However,  it  is  noted  that  there  is  no  fully  accepted  formal  model  that 
describes  the  factors  comprising  the  concept  of  workload  nor  is  there  a  model  that  describes  the 
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contribution  of  individual  factors  to  the  overall  workload  concept.  Three  broad  categories  of 
workload  definitions  have  been  presented  as  (Lysaght  et  ah,  1989): 

•  The  amount  of  work  and  number  of  tasks  to  be  perfonned. 

•  The  aspect  of  time  that  the  operator  is  concerned  with. 

•  The  subjective  psychological  experiences  of  the  operator. 

An  assessment  of  potential  operator  workload  is  one  of  the  major  considerations  in  any 
human  based  system  (Williges  and  Williges,  1981).  However,  the  concept  of  mental  workload 
has  yet  to  achieve  a  consistent  definition  in  the  research  community.  Part  of  the  problem  is  that 
workload  is  both  task  specific  and  individual  specific  (Rouse  et  al.,  1993).  A  start  is  the 
definition  is  that  workload  is  the  capacity  to  perform  (Fallesen  and  Quinkert,  1990).  Another 
definition  is  the  cost  incurred  by  the  human  operator  in  accomplishing  the  imposed  task 
requirements  and  involves  both  physical  and  mental  activities  (AGARD,  1998).  This  workload 
cost  is  indicative  of  the  combined  effects  of  the  demands  imposed  by  the  tasks  themselves,  the 
information  and  equipment  used,  the  task  environment,  operator  skills  and  experience,  operator 
strategies,  the  effort  expended  in  perfonnance  of  the  work,  and  the  emotional  response  to  the 
situation.  There  are  many  accepted  methods  for  measuring  human  physical  and  mental 
workload,  (Hamilton  et  al.,  1991;  Hendy,  Hamilton,  and  Landry,  1993;  Kumashiro,  1995;  Reid  et 
al.,  1986;  Tijerina,  Kiger,  Rockwell,  and  Tomow,  1995;  Vidulich,  Ward,  and  Schueren,  1991; 
Wilson  and  Eggemier,  2001;  Young  and  Stanton,  2001)  to  quote  only  a  small  sampling  from  the 
literature. 

However,  methods  for  predicting  mental  workload  is  less  precise,  owing,  in  part  to  the 
absence  of  a  consistent  definition  of  what  constitutes  mental  workload.  The  general  aim  of 
workload  prediction  techniques  is  to  predict  accurately  the  relationship  between  operator  task 
demands  and  the  capacity  of  the  operator  (AGARD,  1998).  The  issue  is  whether  changes  in 
performance  can  be  predicted  given  the  characteristics  of  an  individual  task  or  the  relationships 
between  multiple  tasks.  This  fonn  of  workload  prediction  is  generally  performed  using  a  task 
analysis  technique  usually  implemented  in  a  discrete  event  computer  simulation  environment 
such  as  Micro  Saint™  (MA&D,  1996).  Another  form  of  workload  analysis  is  mission  timeline 
analysis,  which  computes  the  ratio  between  time  available  and  time  required  to  perfonn  the  task. 
A  ratio  greater  than  1.0  implies  that  the  work  tasks  cannot  be  performed.  Values  of  between  0.85 
and  1.0  are  considered  to  be  indicative  of  workload  problems  (AGARD,  1998). 
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Workload  assessment  is  the  topic  of  extensive  discussion  in  the  literature,  (reference  such 
examples  as  (Gopher  and  Donchin,  1986a;  Hancock  and  Caird,  1993;  Hancock  and  Meshkati, 
1988;  Hart  and  Wickens,  1993;  Hendy  et  ah,  1993;  Rouse,  Edwards,  and  Hammer,  1992;  Rouse 
et  ah,  1993;  Whitaker,  Oatman,  and  Shank,  1987;  Wickens,  1995;  Wierwille  et  al,  1986;  Xie, 
1997))  but  even  today  there  is  no  generally  accepted  paradigm  for  representing  it  in  human 
performance  models.  In  the  paragraphs  that  follow,  I  will  examine  the  concept  as  to  its  meaning 
and  use  for  the  general  application  of  predictive  modeling  of  human  performance  systems  that  is 
typically  carried  out  on  the  computer  using  discrete  event  simulation  techniques.  All  of  the 
example  applications  that  will  be  presented  are  implemented  in  the  commercial  programming 
language  Micro  Saint™,  developed  by  Micro  Analysis  and  Design  Corporation  in  Boulder, 
Colorado.  It  appears  that  Micro  Saint™  has  become  the  de  facto  programming  environment  of 
choice  for  human  factors  research.  In  my  opinion  this  is  primarily  due  to  the  human  factors 
background  of  its  developers  (Laughery  and  Corker,  1997)  as  there  are  many  other  competing 
products  in  the  discrete  event  programming  world.  Some  of  the  better  known  “general  purpose” 
languages  include  Arena,  AweSim,  GPSS,  MODSIM  III,  Simple++,  and  ProModel,  in  addition 
to  Micro  Saint™  (Law,  1997).  The  primary  application  area  of  the  majority  of  these  and  other 
languages  like  them  is  in  the  area  of  manufacturing  and  process  control.  While  Micro  Saint™  is 
also  touted  as  a  general  purpose  discrete  event  language,  and  has  been  used  for  other  applications 
such  as  process  manufacturing  simulation  (Tan,  1991),  its  primary  focus  is  in  the  area  of  human 
performance  modeling. 

One  set  of  criteria  for  selection  of  workload  assessment  techniques  (O’Donnell 
and  Eggemeier,  1994)  includes  the  measures  of  sensitivity,  diagnosticity,  intrusiveness, 
implementation  requirements,  and  operator  acceptance.  These  factors  are  summarized  in 
Table  5. 

Table  5  -  Criteria  For  Selection  Of  Workload  Assessment  Techniques 

(from:  (O'Donnell  and  Eggemeier,  1994)) 


Criterion 

Explanation 

Sensitivity 

Capability  of  a  technique  to  discriminate  significant  variations  in  the  workload  imposed 

by  a  task  or  group  of  tasks 

Diagnosticity 

Capability  of  a  technique  to  discriminate  the  amount  of  workload  imposed  on  different 

operator  capacities  or  resources  (e.g.,  perceptual  versus  central  processing  versus 
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motor  resources) 

Intrusiveness 

The  Tendency  for  a  technique  to  cause  degradations  in  ongoing  primary  task  performance 

Implementation 

Factors  related  to  the  ease  of  implementing  a  particular  technique.  Examples  include 

Requirements 

instrumentation  requirements  and  any  operator  training  that  might  be  required. 

Operator  Acceptance 

Degree  of  willingness  on  the  part  of  operators  to  follow  instructions  and  actually  utilize  a 
particular  technique 

In  1992  a  dissertation  was  presented  (Moscovic,  1992)  that  attempted  the  development 
and  validation  of  a  methodology  to  incorporate  a  predetermined  time  system  (PTS)  and  the 
cognitive  workload  metric  SWAT  (Tsang  and  Wilson,  1997)  into  a  model  of  human  performance 
for  making  decisions  while  using  interactive  display  consoles.  This  model  was  implemented  in 
Micro  Saint™  and  was  a  display  console  interaction  task  network  simulation  using  a  work 
measurement  technique  called  MODular  Arrangement  of  Predetermined  time  Standards 
(MOD  APS).  MOD  APS  is  based  upon  the  PTS  methodology  and  is  a  work  measurement 
technique  that  embodies  44  elements  of  human  body  movements  in  addition  to  numerous 
activities  and  consists  of  a  set  of  databases  containing  standard  time  performance  values  for 
human  body  movements.  There  are  other  PTS  systems  such  as  the  Methods  Time  Measurement 
(MTM)  system  based  on  film  time  studies  of  various  industrial  jobs  and  Computerized  Maynard 
Operation  Sequence  Technique  (CMOST)  where  work  is  measured  by  the  movement  of  objects. 
However,  the  conclusion  was  that  MOD  APS  was  the  most  effective  of  these  approaches  for 
assigning  task  completion  times  for  use  in  the  Micro  Saint™  model.  As  Moscovic  did  not 
actually  provide  a  name  for  her  model,  here  it  will  be  referred  to  as  the  Moscovic  Display  Model 
(MDM)  for  ease  of  reference. 

Moscovic ’s  aim  was  to  develop  the  MDM  methodology  incorporating  the  MOD  APS  PTS 
along  with  the  SWAT  workload  metric  into  a  model  capable  of  predicting  human  performance  in 
this  work  setting.  Findings  from  the  MDM  effort  indicated  that  it  was  a  valid  way  to  predict 
time  performance  and  provided  a  strong  indicator  of  workload,  however,  statistically,  it  did  not 
provide  an  exact  prediction  of  workload  scores  compared  to  SWAT  observations  of  actual 
operators.  Complicating  the  issue  was  the  observation  that  regression  analyses  of  SWAT  scores 
from  two  test  groups  indicated  that  they  were  not  homogeneous  with  respect  to  the  workload 
ratings.  It  is  hard  to  validate  a  computer  model  when  its  baseline  is,  itself,  not  conclusive. 
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An  evaluation  of  some  of  the  issues  surrounding  the  difficulty  in  workload  estimation 
starts  with  an  examination  of  what  workload  is  and  how  it  can  be  measured  in  real  world 
situations.  One  definition  of  workload  is  that  it  is  an  expression  of  the  degree  of  qualitative  and 
quantitative  load  induced  by  work  and  is  closely  related  to  stress  and  industrial  fatigue 
(Kumashiro,  1995).  Mental  workload  is  that  load  placed  on  the  cognitive  capabilities  such  that  it 
taxes  the  information  processing  capabilities  of  human  operators.  Another  definition  of  mental 
workload  states  that  it  is  an  intervening  variable  that  may  be  viewed  as  the  difference  between 
the  capacities  of  the  infonnation  processing  system  that  are  required  for  task  performance  to 
satisfy  perfonnance  expectations  and  the  capacity  available  at  any  given  time  (Gopher  and 
Donchin,  1986b).  The  definition  of  an  intervening  variable,  as  opposed  to  a  hypothetical 
construct,  as  provided  by  MacCorquadale  and  Meehl  in  1948,  discussed  it  as  a  theoretical 
concept  as: 

. . .  simply  a  quantity  obtained  by  a  specified  manipulation  of  the 
values  of  empirical  variables;  it  will  involve  no  hypothesis  as  to 
the  existence  of  unobserved  entities  or  the  occurrence  of 
unobserved  processes;  it  will  contain,  in  its  complete  statement  for 
all  purposes  of  theory  and  prediction  no  words  which  are  not 
defined  either  explicitly  or  by  reduction  sentences  in  terms  of  the 
empirical  variables. 

A  workload  measurement  procedure  is  one  in  which  an  attempt  is  made  to  characterize 
the  conditions  under  which  task  demands  can  or  cannot  be  met  by  the  operator.  A  workload 
measure  is  one  by  which  the  latter  differences  are  expressed  in  relation  to  the  overall  ability  of 
the  human  processing  system  to  process  information  and  generate  responses.  Three  categories  of 
workload  measures  are  presented  as  (Gopher  and  Braune,  1984): 

•  Where  demands  are  expressed  in  terms  of  the  objective  parameters  of  tasks. 

•  Measures  of  response  (behavioral  or  physiological). 

•  Subjective  appraisal  given  by  the  perfonner  to  the  load  he  or  she  experiences 
during  task  perfonnance. 

The  key  appears  to  be  its  effect  on  the  information  processing  abilities  of  the  human 
mind.  Moscovic’s  three  categories  of  mental  workload  measures,  which  have  some  similarities 
to  the  three  above  are: 

•  Psychophysiological  measures  are  based  on  the  premise  that  states  of  cognitive 
workload  can  be  inferred  from  physiological  conditions  such  as  sinus  arrhythmia 
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(SA)  or  heart  rate  where  the  heart  rate  variability  decreases  as  the  cognitive 
workload  increases.  Its  primary  advantage  is  objectivity. 

•  Performance  measures  assess  impairments  to  performance  levels  of  such  things  as 
primary  and  secondary  task  performance  and  provide  immediate  and  direct 
responses  that  are  objective. 

•  Subjective  measures  assess  the  conscious  experience  of  the  operator  through  self 
report  estimates  from  the  operator  and  can  provide  the  most  valid  and  sensitive 
way  to  tap  cognitive  workload.  Techniques  include:  1)  unidimensional  numerical 
ratings;  2)  multidimensional  evaluations;  3)  rank  ordering  of  tasks;  4)  task 
specific  protocols  and  checklists;  5)  stereo  tape  recorder  monitoring.  Examples 
include: 

o  Cooper-Harper  Scales. 

o  NASA-TLX. 

o  Subjective  Workload  Assessment  Technique  (SWAT). 

It  is  important  to  select  a  categorization  of  workload  measures  which  groups  the  various 
workload  techniques  in  a  logical  way  so  that  conflicts  and  discrepancies  on  workload  concepts  is 
minimized  (Williges  and  Wierwille,  1979). 

Fourteen  behavioral  workload  measures  have  been  identified  (Williges  and  Wierwille, 
1979)  and  are  summarized  as: 

1 .  Rating  scales: 

•  Advantages  -  is  a  sensitive  measure  of  workload  and  results  in  little  intrusion  on 
the  primary  task. 

•  Disadvantages  - 

o  Some  approaches  fail  to  follow  rigorous  psychometric  procedures  in  the 
development  of  the  workload  scale. 

o  Confusion  over  the  distinction  over  physical  and  mental  workload. 

o  Respondent  ratings  can  vary  according  to  mental  state,  experience, 
learning  and  natural  abilities. 

o  Respondent  may  simply  not  be  aware  of  the  degree  of  mental  loading  of  a 
given  task. 

2.  Interviews  /  questionnaires: 

•  Advantages  -  Used  as  a  supplemental  measure  in  workload  assessment. 
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Disadvantages  -  Results  cannot  be  given  a  high  priority  because  the  data  can  only 

be  used  in  a  supportive  way  in  workload  assessment. 

Task  Component  /  Time  Summation: 

Advantages  -  Can  be  used  in  an  activity  analysis  fonnat. 

Disadvantages: 

o  The  basic  assumption  of  a  constant  workload  capacity  could  cause  a  bias 
in  the  results. 

o  The  more  remote  the  actual  application  is  from  the  laboratory  where  the 
results  are  measured  the  more  inaccurate  the  workload  assessment  is  likely 
to  be. 

Information  /  Theoretic: 

Advantages  -  Used  for: 

o  Applications  for  visual  monitoring. 

o  Applications  for  continuous  tracking. 

o  Applications  for  complex  information  processing  activities. 

Disadvantages  (at  least  as  of  1979)  — 

o  The  theoretical  fonnulations  underlying  these  procedures  need  further 
development. 

o  Only  a  limited  amount  of  validation  data  are  available  to  support  these 
procedures. 

Nonadaptive,  Arithmetic  /  Logic: 

Advantages  -  Useful  for  monitoring,  shadowing,  mental  math,  memory,  choice 

reaction  time,  auditory  detection,  simple  reaction  time,  problem  solving,  random 

sequence  generation,  and  classification  tasks. 

Disadvantages  - 

o  The  most  difficult  aspect  of  the  secondary  task  methodology  for  assessing 
workload  is  intrusion. 

o  Another  problem  in  workload  estimation  by  the  secondary  task 

methodology  is  the  underlying  assumption  of  task  regularity  or  stationary, 
which  assumes  that  the  primary  task  is  uniform  during  the  analysis  period. 

Nonadaptive,  Tracking: 

Advantages  - 

o  Allows  the  validation  and  development  of  pilot  models  of  workload. 

o  Allows  the  analysis  of  concurrent  workload  involving  monitoring 
functions. 


Disadvantages 


o  Data  does  not  exist  (as  of  1979. . .)  for  real  world  tasks,  thus  this  method 
valid  for  laboratory  use  only. 

o  Extra  tracking  tasks  in  performance  studies  of  actual  task  perfonnance  can 
lead  to  safety  issues  for  operator  in  the  study. 

7.  Time  Estimation: 

•  Advantages  -  Can  be  used  to  imply  the  attention  demands  required  for  task 
performance. 

•  Disadvantages  - 

o  The  display  of  time  information  can  have  a  pronounced  effect  on  primary 
task  performance. 

o  Only  relative,  not  absolute,  workload  assessment  evaluations  can  be  made. 

o  Post  hoc  interpretations  can  vary  considerably  depending  on  the  mode  of 
production  assumed. 

8.  Adaptive,  Arithmetic  /  Logic: 

•  Advantages  - 

o  Tests  show  that  differences  in  the  adaptive  tasks  are  sufficiently  large  to 
demonstrate  changes  in  primary  task  workloads. 

o  Cross  -  adaptive  procedures  eliminate  the  intrusion  of  the  secondary  task 
on  primary  task  perfonnance. 

•  Disadvantages  -  Technique  probably  limited  to  laboratory  and  flight  simulator 
situations 

9.  Adaptive  /  Tracking: 

•  Advantages  -  Can  be  used  as  a  critical  tracking  task  as  a  secondary  task  measure 
that  is  useful  as  a  sensitive  measure  of  varying  levels  of  primary  task  workload. 

•  Disadvantages  -  Useful  in  laboratory  environments  but  have  limited  capability  in 
actual  performance. 

10.  Occlusion:  This  is  a  time-sharing  technique  that  is  similar  to  the  secondary  task 
method. 

•  Advantages  - 

o  Useful  for  studying  attention  demand, 
o  Quite  sensitive  to  control  task  difficulty  and  operator  skill, 
o  Primarily  applicable  to  simulation  research 

•  Disadvantages  - 

o  Safety  in  actual  task  performance  situations. 
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o  Method  not  particularly  sensitive  and  intruded  more  when  compared  to 
other  techniques. 

o  Substantial  lengths  of  time  may  be  necessary  for  operators  to  learn  to  use 
the  occlusion  apparatus. 

1 1 .  Handwriting  Analysis: 

•  Advantages  - 

o  Used  as  a  secondary  task. 

o  Reductions  possible  in  handwriting  legibility  and  sentence  structure  as  a 
function  of  “distraction  stress”. 

•  Disadvantages  - 

o  Requirement  for  a  dedicated  writing  hand, 
o  Environment  must  be  nearly  vibration  and  acceleration  free, 
o  Operator’s  hand  must  be  available  and  ungloved. 

12.  Single  Measures  (Primary  Task): 

•  Advantages  - 

o  Often  used  as  a  means  of  validating  other  workload  measures. 

o  Performance  on  primary  task  as  a  means  of  examining  the  effect  of  the 
secondary  task  to  assess  workload. 

o  Takes  advantages  of  implications  regarding  workload  and  its  relationship 
to  primary  task  measures. 

•  Disadvantages  - 

o  While  high  workload  situations  (near  operator  overload)  are  discernable 
by  primary  task  measures,  low  workload  situations  may  not  be. 

o  Measurement  of  primary  task  variables  tends  to  be  complicated. 

13.  Multiple  Measures  (Primary  Task): 

•  Advantages  -  Measurement  of  workload  in  multiple  task  environments. 

•  Disadvantages  -  The  more  variables  that  are  measured  means  that  some  will  not 
change  reliably  as  a  function  of  workload. 

14.  Mathematical  Modeling: 

•  Advantages  -  Useful  for  areas  of  human  operator  decision  processes,  supervisory 
processes,  team  interactions,  and  operator  workload. 

•  Disadvantages  - 

o  Usually  confined  to  specific,  well  constrained,  and  perhaps  repetitive, 
human  operator  tasks. 
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o  As  higher  mental  processes  become  more  involved,  modeling  techniques 
tend  to  be  less  applicable. 

This  is  but  one  example  of  attempts  to  categorization  of  workload  measures.  There  are 
others.  While  there  is  mutual  agreement  over  the  importance  of  mental  workload,  there  is 
substantial  controversy  over  the  best  type  of  workload  measurement.  It  is  argued  that  all  three 
forms  of  measures  are  needed  in  any  workload  measurement  attempt,  however,  practical 
limitations  make  this  unfeasible.  Further,  the  controversy  over  the  best  type  of  workload 
measure  for  actual  evaluation  of  real  human  perfonnance  complicates  the  issue  of  how  to  attempt 
to  predict  it  in  a  computer  simulation. 

Attempting  to  put  some  form  to  this  effort,  Moscovic  turned  to  task  network  modeling 
and  Laughery’s  Micro  Saint™  simulation  tool  (Laughery,  1989a).  Regardless  of  the  ongoing 
theoretical  debate  on  how  to  measure  it  and  what  it  actually  is,  researchers  have  and  are 
continuing  to  build  on  the  knowledge  of  what  both  physical  and  mental  workload  constitute. 
Laughery  proposed  that  the  theoretical  aspects  of  the  operator  approach  to  workload  analysis 
consist  of  a  combination  of  task  network  modeling  and  the  multiple  resource  theory  (Wickens, 
Sandry,  and  Vidulich,  1983). 

Task  network  modeling  is  a  technique  where  human  performance  is  decomposed  into  a 
series  of  sub  functions,  which  are  then  subsequently  decomposed  into  tasks,  which  are 
represented  in  a  task  network.  It  is  noted  that  task  network  modeling,  in  and  of  itself,  is  not 
inherently  a  model  of  human  workload  but  provides  output  that  is  the  time  required  to  perform  a 
set  of  tasks  and  the  sequence  in  which  the  tasks  are  perfonned.  However,  a  promising  theory  of 
operator  workload  which  is  consistent  with  task  network  modeling  is  Wicken’s  multiple  resource 
theory  (Laughery,  1989a).  The  multiple  resource  theory  (Wickens  et  ah,  1983)  suggests  that 
humans  have  more  than  just  one  infonnation  processing  source  that  can  be  accessed  singly. 
Rather,  they  have  several  resources,  which  can  be  accessed  simultaneously.  Laughery 
summarizes  his  comments  on  human  workload  by  listing  his  set  of  issues  associated  with  the 
evaluation  of  workload: 

•  What  are  the  channels  of  workload? 

•  How  should  attention  demands  within  a  workload  channel  be  combined  across 
tasks? 

•  How  should  attention  demands  be  combined  across  workload  channels? 
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•  What  values  for  workload  represent  "excessive"  workload? 

•  What  about  task  dumping? 

Thus,  parlaying  on  Laughery’s  comment  that  excessive  human  workload  is  not  usually 
caused  by  one  particular  task  required  of  the  operator,  but  rather  is  a  result  of  the  human  having 
to  perfonn  several  tasks  simultaneously  that  leads  to  overload,  Moscovic  used  the  multiple 
resource  theory  as  the  theoretical  approach  to  evaluating  workload  and  the  visual,  auditory, 
cognitive  and  psychomotor  (VACP)  (McCracken  and  Aldrich,  1984)  technique  for 
characterization  of  workload  demand  as  a  representation  of  human  infonnation  processing.  This 
technique  states  that  each  operator’s  activity  in  a  task  network  is  characterized  by  the  workload 
demand  required  in  each  of  four  channels.  These  channels  are  the  auditory  channel,  the  visual 
channel,  the  cognitive  processing  channel,  and  the  psychomotor  output  channel. 

Unfortunately,  Moscovic  was  not  able  to  claim  complete  success  with  her  MDM 
methodology.  While  the  PTS  /  MOD  APS  approach  did  provide  a  successful  means  to  predict 
task  time  perfonnance,  the  effort  to  generate  a  regression  equation  based  predictive  model  of 
human  workload  based  on  an  analysis  of  SWAT  data  was  not.  She  states  that  the  findings 
indicate  that  either  the  methodology  of  detennining  workload  was  not  adequate  or  that  the 
groups  were  heterogeneous  in  terms  of  workload  ratings.  She  noted  that  the  small  number  of 
participants  in  each  empirical  group  could  have  had  an  impact  on  the  results. 

This  finding,  however,  did  not  dampen  the  enthusiasm  for  the  approach,  which  is  an 
enthusiasm  that  I  endorse  and  also  have  as  a  personal  research  goal.  The  utility  of  using 
regression  equation  techniques  in  predictive  models  that  use  beta  weights  for  variables  as  a 
means  of  workload  projection  and  prediction  is  just  too  powerful  to  ignore.  A  technique  known 
as  the  Subjective  Workload  Dominance  (SWORD)  (Vidulich  et  al.,  1991)  method  has  been  used 
in  studies  to  project  workload.  The  SWORD  technique  has  human  operators  assign  retrospective 
ratings  from  an  abstract  workload  scale  to  a  task  without  comparison  to  other  tasks.  The  three 
steps  in  the  SWORD  technique  are  collecting  the  raw  judgment  data,  constructing  the  judgment 
matrices,  and  calculating  the  SWORD  ratings.  As  the  SWORD  technique  is  an  expert  opinion 
extraction  technique  it  is  hypothesized  that  it  could  be  useful  as  a  projective  workload  tool. 

Tests  have  shown  that  it  can  be  used  as  a  predictive  tool  provided  SME  opinion  is  available  to 
establish  the  required  workload  parameters. 
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In  1995  another  dissertation  was  presented  (Green,  1995)  that  introduced  a  variant  on 
traditional  task  analysis  called  the  Perfonnance  Assessment  Methodology  (PAM).  This 
methodology  was  developed  that  utilized  observable  events  and  actions  along  with  inferred 
decisions  to  describe,  assess,  and  predict  perfonnance  of  operators  in  a  complex  human  -  system 
-  task  perfonnance  situation.  PAM  consisted  of  a  new  task  analysis  methodology  that  was 
developed  to  generate  objective  information  from  human  performance  involving  complex  tasks. 
SME  opinion  was  used  to  break  down  each  complex  task  into  observable  elements  along  with 
unobservable  elements  that  were  infened  to  have  occuned  in  support  of  the  observable  events. 
Guidelines  were  developed  to  specify  the  level  of  detail  for  the  task  analysis  breakdown.  From 
this  information  a  simulation  model  was  constructed  using  Micro  Saint™  that  represented  the 
task  elements.  Possibly  concurrent  with  the  development  of  the  computer  simulation  was  the 
observation  of  operators  trained  in  the  task(s)  being  investigated,  which  resulted  in  objective  data 
on  their  actions  that  could  be  observed.  Data  from  these  observations  then  was  used  as  numeric 
input  to  the  computer  simulation.  The  simulation  was  then  run  and  the  results  compared  to  the 
real  events  that  were  observed.  Following  these  initial  runs  the  model  was  altered  to  produce 
results  similar  to  what  was  observed  during  the  data  collection  phase.  This  process  is  stated  to 
have  produced  a  model  capable  of  accurately  reflecting  human  perfonnance  on  the  task. 

The  key  to  the  PAM  methodology,  and  the  thing  that  sets  it  apart  from  other  similar 
techniques,  appears  to  be  the  breakdown  of  the  complex  tasks  into  first,  observable  elements,  and 
second,  unobservable  elements  that  were  predicted  to  have  taken  place  in  support  of  the  observed 
task  elements.  Critical  to  this  analysis  is  the  participation  of  proficient  SMEs  capable  of  making 
these  assessments.  Once  the  PAM  model  was  validated  against  the  observable  performance 
parameters  then  it  could  be  used  for  purposes  that  are  ascribed  to  computer  simulations  in 
general  which  is  to  investigate  performance  conditions  under  different  task  environments  that 
may  or  may  not  be  observable  in  the  real  world.  In  other  words,  if  a  computer  model  is  accepted 
for  its  ability  to  produce  a  realistic  simulation  under  one  given  set  of  circumstances  then  it  has  a 
probable  ability  to  produce  realistic  results  for  other  perfonnance  circumstances.  Unfortunately, 
the  farther  the  investigative  domain  drifts  from  the  validated  situation,  the  less  likely  the 
simulation’s  results  can  be  accepted.  However,  risk  assessment  and  acceptance  is  another  topic 
completely  and  deviates  from  the  topic  here. 
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Although  workload  is  not  directly  addressed  in  this  report,  it  is  implied  through  the  task 
analysis  and  resulting  human  performance  measures.  However,  the  Micro  Saint™  model  used  in 
this  study  was  only  capable  of  predicting  task  perfonnance  time  thereby  providing  a  “taskload” 
prediction  or  component  of  workload.  The  taskload  times  in  this  case  were  the  overall  functional 
recognition  time  values  observed  in  the  model.  These  results  mirror  the  Moscovic  tests  where 
the  modeling  accurately  reflected  the  temporal  component  of  workload  as  it  simulated  the 
operator  perfonnance  during  functional  tests.  The  functional  recognition  task  that  Green  chose 
to  evaluate  the  operator  performance  where  the  operators  were  required  to  determine  the  purpose 
of  the  information  on  the  display  required  operators  to  process  both  auditory  and  visual 
information.  His  finding  validated  the  task  time  predictive  nature  of  the  simulation  where  there 
was  a  high  correlation  between  the  Micro  Saint™  predictions  and  observed  operator  performance 
of  the  functional  recognition  task. 

Two  years  later  an  updated  PAM  methodology  was  applied  to  the  relationships 
describing  visual  events,  decisions,  and  actions  while  operating  a  visual  display  system  (Orrell, 
1997).  In  this  case  PAM  provides  a  quantitative  basis  for  evaluating  display  image  quality  based 
on  the  visual  events  that  occur  in  a  task  and  is  defined  as  a  framework  that  shows  the 
relationships  among  operator  tasks,  display  system  parameters,  and  perfonnance  measures  for 
these  parameters.  With  the  PAM  methodology,  operator  tasks  are  decomposed  into  events, 
decisions,  and  actions.  When  a  triggering  event  occurs,  the  operator  makes  a  decision  and  then 
responds  with  an  action.  Thus,  observable  perceptions  or  events  lead  to  decisions  and  actions.  A 
task  might  have  many  events,  decisions,  and  actions  associated  with  it. 

In  this  application  PAM  is  based  on  a  network  analysis  model  of  time  sequential  events 
and  is  an  aggregate  measure  of  performance  (e.g.,  time,  errors,  and  workload)  for  individual  task 
parameters.  It  allows  consideration  of  any  human  performance  measure  including  accepted 
measures  such  as  speed  (time)  and  accuracy  (errors).  Stress  or  workload  provides  an  additional 
measure  of  performance  and  is  defined  as  the  difference  between  the  perceived  demands  of  the 
task  and  a  person’s  perceived  capacity  to  cope  when  coping  is  important. 

Orrell  refines  the  previous  definition  of  PAM  as  a  framework  that  shows  the  relationships 
among  operator  tasks,  system  parameters,  and  performance  measures  for  those  parameters. 

PAM  decomposes  the  operator  tasks  into  events,  decisions  and  actions  (EDA)  and  provides  a 
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framework  for  describing  the  relationships  between  them.  When  events  occur,  the  operator 
makes  a  decision  and  then  responds  with  an  action.  A  task  might  have  many  events,  decisions, 
and  actions  associated  with  it.  In  the  taxonomy  that  is  developed  for  PAM,  parameters  may  be 
associated  with  many  events,  and  individual  decisions  and  actions  can  be  isolated  for  further 
evaluation.  Thus,  a  parameter  may  be  related  to  any  level  of  an  EDA  so  that  multiple  decisions 
may  use  the  same  parameter.  The  evaluation  may  start  with  actions  and  then  detennine  how 
events  and  decisions  influence  these  actions.  For  operator  tasks  PAM  shows  the 
interrelationships  between  events,  how  system  parameters  influence  individual  events,  and  how 
these  parameters  are  related  to  performance  measures.  This  sequence  is  diagrammed  in  Figure 
11. 


Figure  11  -  PAM  Task  Fevel  Architecture 
(Adapted  From:  (Orrell,  1997)) 

The  influence  of  system  parameters  is  determined  for  each  task  relevant  event. 
Functional  relationships  between  performance  measures  and  parameters  are  then  detennined.  A 
PAM  efficiency  measure  is  created  by  combining  and  weighting  performance  scores  across  like 
parameters.  Thus,  efficiency  is  an  aggregate  measure  of  performance  according  to  time,  errors 
and  workload  for  individual  system  parameters.  Any  human  performance  measure  can  be  used 
with  PAM  including  such  measures  as  speed  (time)  and  accuracy  (errors).  Workload  is 
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measured  according  to  stress  and  is  defined  as  the  difference  between  the  perceived  demands  of 
the  task  and  a  person’s  perceived  capacity  to  cope  when  coping  is  important. 

When  setting  up  the  database  there  are  two  elements  that  require  definition  SMEs.  These 
are  task  relevant  events  and  system  parameters.  A  ranking  and  rating  methodology  is  used  to 
determine  the  important  events  and  parameters.  With  this  procedure  each  factor  is  ranked  in 
order  of  importance  within  its  category  on  a  scale  of  0  to  1  with  ties  permitted.  The  factors  with 
the  highest  ratings  are  then  selected  for  inclusion  in  the  appropriate  task  where  the  SME 
determine  the  cutoff  scores. 

A  visual  theoretical  model  called  the  Model  of  Visual  Events  (MOVE)  was  used  to 
describe  the  relationships  between  visual  events,  decisions,  and  resulting  actions  and 
implemented  the  PAM  methodology  for  this  visual  display  application.  MOVE  was  developed 
to  describe  categories  of  perceptual  decisions  that  are  associated  with  visual  events.  These 
categories  included  the  ability  to  detect,  identify,  discriminate,  and  evaluate  visual  objects  or 
targets.  The  purpose  of  the  investigation  was  to  develop  a  quantitative  basis  for  image  quality 
evaluation.  The  PAM  top-level  architecture  previously  shown  in  Figure  1 1  is  applied  to  the 
visual  display  application  to  create  the  MOVE  model.  In  this  application,  MOVE  expands  on  the 
EDA  model  by  incorporating  the  components  of  perception,  decision-making,  and  response  / 
action  sequence.  In  MOVE  the  flow  of  the  model  is  dependent  on  the  nature  of  the  task.  For 
simple  tasks  the  image  would  be  immediately  detected  and  evaluated.  However,  for  more 
complex  tasks  the  actions  first  detects,  identifies  or  recognizes,  discriminates  and  then  interprets 
or  evaluates. 

Summarizing  the  use  of  PAM,  the  preparation  stages  for  this  methodology  include  a 
general  task  analysis  and  an  event  analysis  to  be  conducted  with  SME  assistance.  Using  the 
PAM  based  model  includes  the  steps  of  calculating  the  event  efficiency  across  all  the  individual 
parameters,  calculating  the  task  efficiency  for  each  task  completion  pathway,  and  calculating  the 
network  efficiency. 

While  Orrell  claims  workload,  and  synonymously,  stress  as  a  measure  of 
performance  there  is  no  direct  mention  of  how  he  measures  or  attempts  to  predict  it. 

While  he  defines  it  as  the  difference  between  the  perceived  demands  of  the  task  and  a 
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person’s  perceived  capacity  to  cope  with  it,  it  is  only  assumed  that  it  must  be  a  part  of  the 
task  and  network  efficiency  calculations. 

A  multi-year  effort  conducted  by  the  Human  Research  and  Engineering  Directorate  of  the 
U.S.  Army  Research  Laboratory  was  the  Computer  Modeling  of  Human  Operator  System  Tasks 
(CoHOST)  project  (Middlebrooks  et  ah,  1999a;  Middlebrooks  et  ah,  1999b).  This  project  also 
provided  the  computer  simulation  model  for  a  followon  master’s  thesis  (Middlebrooks,  2001). 
This  project  was  undertaken  with  the  objective  to  investigate  potential  effects  on  human  mental 
and  physical  performance  capabilities  during  combat  operations  from  the  introduction  of  a  new 
command  and  control  vehicle  equipped  with  modernized  digital  communications  systems.  The 
objective  of  the  project  was  to  produce  a  task  perfonnance  and  workload  model  for  a  maneuver 
battalion  task  force  using  this  mobile  computer  communications  system  using  the  discrete  event 
programming  environment  in  Micro  Saint™.  The  intent  of  the  model  was  to  investigate  the 
efficiency  of  information  flow  and  task  loading  during  the  conduct  of  an  extended  mission  and  to 
compare  soldier  task  and  workload  predictions  in  order  to  answer  the  following  questions: 

•  Is  one  configuration  of  personnel  and  communications  equipment  better  or  worse 
than  another? 

•  Can  the  human  operator  continue  to  function  effectively  during  extended  periods 
of  on-the-move  operations  in  the  vehicle? 

A  taxonomy  of  human  performance  (Fleishman  and  Quaintance,  1984)  was  chosen  to 
provide  a  qualitative  basis  for  workload  and  task  perfonnance  evaluation  in  the  model.  This 
taxonomy  consists  of  52  human  physical  and  mental  knowledge,  skill,  and  ability  (KSA)  taxons 
that  are  designed  to  be  able  to  describe  human  performance  in  any  generalized  work  setting. 

Fifty  of  the  52  KSAs  were  subsequently  chosen  for  use  with  the  CoHOST  model  as  two  of  the 
auditory  KSAs  were  deemed  to  be  repetitive  of  others  for  the  work  setting  being  simulated.  The 
remaining  50  KSAs  were  grouped  into  eight  cognitive  and  physical  perfonnance  clusters  so  that 
selected  performance  demand  weightings  could  be  applied  to  selected  clusters  to  account  for  the 
specifics  of  this  work  situation.  These  50  KSAs  in  their  respective  cluster  assignments  are 
shown  in  Figure  12. 

A  numerical  database  was  established  that  contained  ratings  on  a  scale  of  from  1  to  7  of 
how  much  each  of  the  taxons  applied  to  work  performance  for  each  job  category  in  the  work 
group  the  simulation  was  representing.  SME  opinion  was  used  to  develop  the  ratings  for  this 
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database  according  to  a  computer  based  questionnaire  using  a  7  point  Likert  like  behaviorally 
anchored  rating  scale. 


Cognitive  Skill  and  Experience  Clusters 


Communication 

1 .  Oral  Comprehension 

2.  Written  Comprehension 

3.  Oral  Expression 

4.  Written  Expression 


Conceptual  (1.20) 

5.  Memorization 

6.  Problem  Sensitivity 

7.  Originality 

8.  Fluency  of  Ideas 

9.  Flexibility  of  Closure 

10.  Selective  Attention 

1 1 .  Spatial  Orientation 

12.  Visualization 


Reasoning  (1 .25) 

13.  Inductive  Reasoning 

14.  Category  Flexibility 

15.  Deductive  Reasoning 

16.  Information  Ordering 

17.  Mathematical  Reasoning 

18.  Number  Facility 


Speed-loaded  (1 .22) 

19.  Timesharing 

20.  Speed  of  Closure 

21.  Perceptual  Speed 
and  Accuracy 

22.  Reaction  Time 

23.  Choice  Reaction  Time 


Cognitive  skills  weighted  in  the  model 


Vision 

24.  Near  Vision 

25.  Far  Vision 

26.  Night  Vision 

27.  Visual  Color 
Discrimination 

28.  Peripheral  Vision 

29.  Depth  Perception 

30.  Glare  Sensitivity 


Perceptual-Motor  Ability  Clusters 


Audition 

31.  General  Hearing 

32.  Auditory  Attention 

33.  Sound  Localization 


Psychomotor 

34.  Control  Precision 

35.  Rate  Control 

36.  Wrist-Finger  Speed 

37.  Finger  Dexterity 

38.  Manual  Dexterity 

39.  Arm-hand  Steadiness 

40.  Multi-Limb  Coordination 


Gross  Motor 

41.  Extent  Flexibility 

42.  Dynamic  Flexibility 

43.  Speed  of  Limb  Movement 

44.  Gross  Body  Equilibrium 

45.  Gross  Body  Coordination 

46.  Static  Strength 

47.  Explosive  Strength 

48.  Dynamic  Strength 

49.  Trunk  Strength 

50.  Stamina 


Fleishman,  E.  A.  and  Quaintance,  M.  K.  (1984)  Taxonomies  of  Human  Performance: 
The  Description  of  Human  Tasks.,  Orlando:  Academic  Press. 


Figure  12  -  Knowledge,  Skills,  and  Abilities  Taxonomy 
(Middlebrooks  et  al.,  1999b) 


The  CoHOST  computer  simulation  used  these  ratings  for  instantaneous  (performed  every  100 
seconds  in  simulation  time)  calculations  of  task  loading  by  summing  up  all  the  KSA  values  that 
were  being  applied  in  the  execution  of  tasks  that  were  being  performed  at  the  time  of  the 
calculation  by  the  simulated  operator.  The  further  summing  of  the  1 00  second  interval  taskload 
calculations  over  the  course  of  selected  time  intervals  and  the  total  scenario  of  the  simulation  run 
produced  what  was  called  a  “workload”  value  for  the  operator  for  the  perfonnance  of  the  tasks 
that  were  conducted  during  the  course  of  the  time  interval.  A  sample  of  the  types  of  tasks  being 
performed  by  the  operators  in  the  workgroup  is  shown  in  Figure  13  and  the  decomposition  of 
these  performance  tasks  into  taxons  is  illustrated  in  Figure  14.  This  decomposition  follows 
procedures  previously  described  for  how  task  based  performance  indices  are  established  for 
inclusion  in  a  task  network  simulation. 
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Thus,  what  was  called  workload  in  the  original  CoHOST  simulation  was  a  distinct 
departure  from  workload  assessment  methodologies  in  the  literature  that  use  such  techniques  as 
SWAT  and  NASA-TLX.  In  fact,  it  caused  enough  of  a  controversy  that  the  term  was  changed  to 
taskload  in  a  master’s  thesis  on  this  subject  (Middlebrooks,  2001). 


Input  Process  Output 


Figure  13  -  CoFIOST  Task  Flow  Sequence 
(Middlebrooks  et  al.,  1999b) 


The  issue,  then,  is  how  can  a  computer  simulation  be  used  to  predict  human  performance 
“workload”?  The  CoHOST  definition  as  the  “performance  of  tasks  over  time”,  in  hindsight,  is 
not  acceptable.  Moscovic  made  what  appears  to  be  a  creditable  attempt  to  incorporate  workload 
predictions  into  her  simulation  using  SWAT  techniques  but,  by  her  own  admission,  came  up 
short.  Even  though  her  experiments  were  inconclusive,  Moscovic  stated  “there  is  merit  to  using 
the  regression  equation  technique  in  a  predictive  model  by  identifying  beta  weights  for  variables 
using  projective  workload  techniques. 

Figure  15  illustrates  Moscovic’s  experimental  findings  that  indicate  that  workload  is  a 
separate  measure  from  performance.  Her  comment  is  that  performance  is  a  metric  of  the  number 
of  errors  committed  over  time  while  the  definition  of  SWAT  scores  is  that  they  are  relative 
ratings  based  on  the  rank  ordering  of  workload  dimensions.  If  performance  does  not  correlate  to 
workload,  then  what  does?  If  workload  is  an  independent  metric  of  human  performance,  then, 
the  question  is  “how  can  it  be  quantitatively  represented?”  Can  algorithmic  predictors  of 
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workload  be  developed  based  on  regression  or  other  techniques?  This  is  a  topic  area  of  interest 
for  future  research. 


Communication 

1.  Oral  Comprehension 

2.  Written  Comprehension 

3.  Oral  Expression 

4.  VWitten  Expression 


Vision 

24.  Near  Vision 

25.  Far  Vision 

26.  Night  Vision 
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28.  Peripheral  Vision 
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30.  Glare  Sensitivity 


Cognitive  Skill  and  Experience  Ousters 


Speed-loaded  (1.22) 

19.  Timesharing 

20.  Speed  of  Closure 

21.  Perceptual  Speed 
and  Accuracy 

22.  Reaction  Time 

23.  Choice  Reaction  Time 

11.  Spatial  Orientation 

12.  Visualization 


Conceptual  (1.20) 

5.  Memorization 

6.  Problem  Sensitivity 

7.  Originality 

8.  Fluency  of  Ideas 

9.  Flexibility  of  Closure 
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Reasoning  (1.25) 

13.  Inductive  Reasoning 

14.  Category  Flexibility 

15.  Deductive  Reasoning 

16.  Information  Ordering 

17.  Mathematical  Reasoning 

18.  Number  Facility 


Cognitive  skills  weighted  in  the  model 


Perceptual-Motor  Ability  Ousters 


Audition 

31.  General  Flearing 

32.  Auditory  Attention 

33.  Sound  Localization 


Psychomotor 

34.  Control  Precision 

35.  Rate  Control 

36.  VWist-Finger  Speed 

37.  Finger  Dexterity 

38.  Manual  Dexterity 

39.  Arm-hand  Steadiness 

40.  Multi-Limb  Coordination 


Gross  Motor 

41.  Extent  Flexibility 

42.  Dynamic  Flexibility 

43.  Speed  of  Limb  Movement 

44.  Gross  Body  Equilibrium 

45.  Gross  Body  Coordination 

46.  Static  Strength 

47.  Explosive  Strength 

48.  Dynamic  Strength 

49.  Trunk  Strength 

50.  Stamina 


Detail  Job  Function 

DutyName 

Itfo  Be  Used  In  The  Model) 

A 

01 -Receive  and  Record/  Analog 

£ 

02-Pass  Information 

A 

03-Listen  -  Receive  Information 

04-Secondary  Monitor 

1 

05-Log  Message 

w 

06-Route  (Outside  the  Section) 

Communicate  and  Report 

07-Send  Message 

x 

08-Verbal  Order 

1 

09-Roll  Up  Reports 

m 

10-Call  to  Conference 

V 

26-Receive  Digital  Message 

\ 

27-Input  Data  Into  Computer 

28-Send  Digital  Information 

11 -Decide  Action 

Decide  and  Recommend  /  Greet 

12-Decide 

13-Recommend  Action 

IIH_\ 

14-Estimate  Impact 

15-Data  Gathering/  Analog 

Equate  and  Estimate  Impact 

29-Data  Gathering/  Digital 

16-Find  Options 

17-Compare  Alternatives 

A  q  rv 

19-Read/  Analog 

30-Read/  Digital 

20-Scan 

21  -Update/  Analog 

22-Check  Status 

Identity/Understand  Situational  Picture 

23-Problem  Definition 

24-Listen  /  Monitor-  Analog 

31- Scan  Digital 

32-  Monitor-  Digital 

Manage  Resources 

25-Manage  Resources 

Figure  14  -  CoHOST  Task  Decomposition  and  Translation  From  Taxonomy 

(Middlebrooks  et  ah,  1999b) 
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MEAN  PERFORMANCE 


MEAN  WORKLOAD  (INVERSE) 


CONFIGURATION 


Figure  15  -  Performance  /  Workload  Comparison 
(Moscovic,  1992) 


There  are  five  interrelated  issues  that  are  associated  with  attempts  to  extend  the 
scope  and  applicability  of  human  performance  models  (HPM)  to  more  complex  problems 
and  designs  (Baron,  Kruser,  and  Huey,  1990): 

1 .  Complex  /  comprehensive  models:  Most  existing  HPMs  have  been 
developed  only  for  relatively  simple  situations. 

2.  Model  parameterization:  As  models  become  more  complex,  the  number 
of  parameters  related  to  human  performance  in  the  model  is  likely  to 
increase. 

3.  Model  validation:  As  models  become  more  complex,  they  also  become 
more  difficult  and  costly  to  validate. 

4.  Underutilization  /  inaccessibility  of  HPMs:  Most  complex  HPMs  have  not 
been  used  widely  or  subjected  to  independent  evaluation. 
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5.  Potential  for  misuse  /  misunderstanding:  As  models  become  more 
complex,  they  also  become  significantly  more  difficult  to  use. 

Three  additional  issues  related  to  modeling  the  future  role  of  operators  in  and  of  complex 

systems  are  (Baron  et  ah,  1990): 

1 .  Accounting  for  mental  aspects  of  tasks:  In  an  attempt  to  deal  with 
cognitive  aspects  of  thee  operator’s  tasks,  there  has  been  increasing 
interest  in  incorporating  aspects  of  the  operator’s  tasks,  there  has  been 
increasing  interest  in  incorporating  mental  models  into  HPMs. 

2.  Developing  and  using  knowledge  based  models:  Along  with  the  increased 
interest  and  popularity  of  artificial  intelligence  (AI),  there  has  been  a  rush 
toward  the  development,  integration,  and  use  of  intelligent  or  knowledge- 
based  models  (or  sub  models). 

3.  Accounting  for  individual  differences:  The  effects  of  individual 
differences  have  been  largely  ignored  in  HPMs  (as  of  1990),  in  favor  of 
using  average  indices  of  human  characteristics  representing  the  ideal,  fully 
trained  operator.  A  conclusion  reached  in  this  dissertation  is  that  this  is 
still  the  case  today. 

An  early  modeling  effort  can  provide  quantitative  and  qualitative  analyses  that  allow 
design  trade-off  studies  to  include  a  variety  of  human  performance  factors  along  with 
other  system  variables  (Baron  et  ah,  1990)  (p. 86).  Amen. 

The  literature  is  rich  with  narrative  comment  on  the  subject  of  the  evaluation  and 
estimation  of  mental  workload,  however,  it  remains  a  topic  that  is  open  to  interpretation 
and  subjective  opinion.  Table  6  provides  a  summary  of  some  of  the  more  common 
subjective  techniques  along  with  the  perfonnance  measure  of  each. 


87 


Table  6  -  Subjective  Evaluation  Methods  for  Workload  Assessment 

(Braun  et  al.,  1999) 


Method 

Technique 

Outcome  Measures 

NASA  Bipolar 
(Vidulich  andTsang,  1985) 

Rating 

Overall  workload,  performance, 
frustration,  task  difficulty,  fatigue 

Multidimensional  bipolar  rating  scale 
(Braun  et  al.,  1999) 

Rating 

Overall  workload,  frustration  level, 
stress,  performance,  fatigue 

NASA-TLX  (NASA-Task  Load  Index) 
(Braun  et  al.,  1999) 

Rating 

Overall  index  of  mental  demand, 
physical  demand,  temporal  demand, 
performance,  effort,  frustration 

SWAT  (Subjective  Workload 

Assessment  Technique) 

(Vidulich  andTsang,  1985) 

Rating 

Time  load,  mental  load,  stress  load 

MCH  (Modified  Cooper  Harper) 
(Wierwille  and  Casali,  1983) 

Rating 

Task  difficulty 

McDonnell  Rating  Scale 
(Braun  et  al.,  1999) 

Rating 

Control  difficulty,  attentional 
demand 

Bedford  Workload  Scale 
(Braun  et  al.,  1999) 

Rating 

Overall  workload,  task  difficulty, 
perceived  exertion 

Borg  Scale 
(Braun  et  al.,  1999) 

Rating 

Perceived  exertion 

2.3.  Methods  and  Procedures  for  Cognitive  Performance  Assessment. 

A  key  factor  in  trying  to  determine  cognition  requirements  for  system  analyses  is  the 
ability  to  estimate  the  cognitive  performance  responses  of  real  human  operators  working  in  real 
work  domains.  There  are  several  approaches  and  methodologies  that  have  been  explored  in 
previous  research  and  proven  to  be  effective  for  certain  applications.  A  set  of  these  approaches 
is  reviewed  here  as  a  description  of  those  approaches  that  are  considered  to  be  useful  in 
formulating  the  COMPASS  paradigm. 

2.3.1.  Evaluation  of  Human  Cognitive  Perfonnance. 

Understanding  and  evaluating  the  various  aspects  of  human  mental  performance  will 
often  begin  with  field  observations  of  individual  and  group  activities  as  they  occur  within  the 
system  domain  being  studied.  Some  of  the  perfonnance  parameters  and  their  definitions  that  can 
be  used  in  observations  to  gather  these  types  of  human  perfonnance  data  include  (Gawron, 
Travale,  and  Neal,  1989): 

1)  Decision  making-the  ability  to  choose  between  two  or  more  alternatives. 


2)  Detection-the  ability  to  discover  or  become  aware  of  a  visual,  auditory,  tactile, 
olfactory,  proprioceptive,  or  kinesthetic  stimulus. 

3)  Fine  manipulation-the  ability  to  manipulate  controls  through  a  physical  effort, 
which  requires  sensitive  movement,  and  touch  rather  than  physical  strength. 

4)  Gross  manipulation-the  ability  to  manipulate  controls  that  require  significant 
physical  strength. 

5)  Numeric  manipulation-the  ability  to  estimate  and  perform  mathematical 
calculations. 

6)  Probability  estimation-the  ability  to  predict  the  chance  of  an  event  occurring. 

7)  Recognition-the  ability  to  identify  a  detected  stimulus. 

8)  Team  coordination- the  ability  to  organize  and  implement  a  team  effort. 

9)  Time  estimation-the  ability  to  predict  how  long  it  will  take  a  moving  body  to 
travel  a  fixed  distance. 

10)  Tracking-the  ability  to  follow  a  moving  target  with  a  control,  e.g.,  joystick. 

Once  these  data  are  collected,  a  traditional  approach  to  providing  input  to  a  quantitative 
analysis  of  this  performance  is  to  use  curve-fitting  techniques  to  develop  regression  equations 
representative  of  the  performance  data.  Types  of  regression  equations  that  can  result  and  their 
forms  are  (Gawron  et  ah,  1989): 

•  Simple  regression:  y  =  mx  +  b. 

•  Simple  regression  with  exponents:  y  =  xm  +  b. 

•  Simple  regression  with  inverse  exponents:  y  =  x  ~m)  +  b. 

•  Simple  regression  with  squared  exponents:  y  =  x(m**2)  +  b. 

•  Simple  regression  with  inverse  squared  exponents:  y  =  x1-'11”2'  +  b. 

•  Multiple  regression:  y  =  mjXi  +  m2X2  +  ...  +  b. 

•  Polynomial  regression:  y  =  mx  +  mx  +  b. 

In  very  large  data  sets  containing  variables  numbering  in  the  dozens,  hundreds,  or  higher, 
these  techniques  have  been  cumbersome  to  attempt.  This  has  helped  to  spawn  alternative, 
though  less  precise,  techniques  such  as  datamining  that  are  supported  by  cluster  analysis  and 
neural  network  evaluations.  However,  the  development  of  high  speed  desktop  computers  and 
statistical  software  that  support  immediate  interactive  analysis  by  the  researcher  now  allow 
regression  approaches  for  these  larger  problems  to  be  considered.  It  is  the  use  of  all  these 
techniques  that  provide  the  best  opportunity  for  success  in  exploratory  evaluations  of  variable 
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interactions  in  large  sets  of  data  such  as  can  be  produced  by  observation  of  human  cognitive 
performance. 

2.3.2.  Ethnography  and  Naturalistic  Observation. 

Ethnographic  research  is  concerned  with  revealing  the  routine  and  the  “paramount 
reality”  of  the  everyday  world  of  individuals  and  groups.  Ethnography  is  defined  as  the  analytic 
descriptions  or  reconstructions  of  intact  cultural  scenes  and  groups.  Ethnographies  recreate  the 
shared  beliefs,  practices,  artifacts,  folk  knowledge,  and  behaviors  of  some  group  of  people. 

What  distinguishes  ethnographic  research  is  its  purpose-which  is  cultural  description. 
Ethnographic  research  seeks  to  build  a  systematic  understanding  of  all  human  cultures  from  the 
perspective  of  those  who  have  learned.  It  stresses  things  like  the  actor’s  understanding  and 
theorizing  about  their  actions  as  opposed  to  traditional  social  science  research  where  the 
researcher  attempts  to  explain  human  action  in  terms  of  psychological  theories  such  as 
attribution  theory.  One  aim  of  ethnographic  research  is  to  record  processes  of  change  rather  than 
stability  (Uzzell,  2000).  It  involves  the  creation  and  ongoing  renegotiations  of  relationships 
between  researchers  and  informants  (Lawlor  and  Mattingly,  2001).  Ethnographies  are  analytic 
descriptions  or  reconstructions  of  intact  cultural  scenes  and  groups  which  delineate  the  shared 
beliefs,  practices,  artifacts,  folk  knowledge,  and  behaviors  of  some  group  of  people  (LeCompte 
and  Goetz,  1982).  One  of  the  most  salient  characteristics  of  qualitative  research,  especially 
ethnography,  is  that  the  researcher  is  preeminently  the  research  tool  (Bonnan,  LeCompte,  and 
Goetz,  1986). 

Some  aspects  of  this  type  of  technique  are: 

•  It  demonstrates  that  the  social  situations  or  context  in  which  action  takes  place  is 
fundamental  to  the  analysis  of  the  behaviour. 

•  Ethnographic  techniques  are  empirical  and  are  almost  without  exception 
employed  in  naturalistic  settings.  The  researcher  is  interested  in  how  individuals 
and  groups  behave  in  their  own  real  world  setting  unmanipulated  by  the 
researcher. 

•  Ethnographic  research  attempts  to  present  the  totality  of  the  phenomenon  under 
investigation. 

Ethnographic  methods  are  used  to  study  people  “in  the  wild,”  as  they  go  about  their 
everyday  activities  in  offices,  homes,  schools,  etc.  The  point  of  ethnography  is  not  to  find  out 
how  people  respond  to  a  constructed  situation  in  which  narrowly  pinpointed  variables  are 
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studied,  as  in  experimental  psychology,  but  to  learn  how  people  actually  work  and  play.  The 
chief  ethnographic  methods  are  interviews,  observations,  and  participant-observation.  One  of  the 
greatest  strengths  of  ethnography  is  its  flexible  research  design.  The  study  takes  shape  as  the 
work  proceeds  (Nardi,  1997b). 

Ethnography  makes  extensive  use  of  qualitative  data  and  one  aim  of  collecting 
ethnographic  data  is  to  assist  in  the  development  and  verification  of  theory  in  order  to  account 
for  human  behavior.  One  of  the  most  salient  characteristics  of  qualitative  research,  especially 
ethnography,  is  that  the  researcher  is  preeminently  the  research  tool  (Bonnan  et  al.,  1986). 
Ethnographers  begin  their  documentation  by  accepting  the  fact  that  what  they  record  in  their 
field  notes  is  already  an  interpretation  of  an  event  (Segall,  1991).  Ethnographic  research 
involves  the  creation  and  ongoing  renegotiations  of  relationships  between  researchers  and 
informants  (Lawlor  and  Mattingly,  2001). 

Ethnographies  are  analytic  descriptions  or  reconstructions  of  intact  cultural  scenes  and 
groups,  which  delineate  the  shared  beliefs,  practices,  artifacts,  folk  knowledge,  and  behaviors  of 
some  group  of  people.  Ethnographic  research  involves  the  acquisition  of  first  hand,  sensory 
accounts  of  phenomena  as  they  occur  in  real  world  settings.  Ethnographers  begin  their 
documentation  by  accepting  the  fact  that  what  they  record  in  their  field  notes  is  already  an 
interpretation  of  an  event  (Segall,  1991).  There  are  three  types  of  data  that  is  provided  by 
ethnographic  research  (LeCompte  and  Goetz,  1982): 

•  Baseline  data-information  about  the  human  and  technological  context  of  the 
research  population  and  program  setting. 

•  Process  data-information  determining  what  happened  in  the  course  of  a  curricular 
program  or  innovation. 

•  Values  data-information  about  the  values  of  the  participants,  the  program 
administrators,  and  the  policymakers  who  financed  the  program. 

Types  of  ethnographic  data  collection  include: 

•  Interactive  methods: 

o  Participant  observation, 
o  Key  informant  interviewing, 
o  Career  histories, 
o  Surveys. 

•  Noninteractive  methods: 
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o  Nonparticipant  observation, 
o  Archival  and  demographic  collection, 
o  Physical  trace  collection. 

The  point  of  ethnography  is  to  find  out  not  how  people  respond  to  a  constructed  situation 
in  which  narrowly  pinpointed  variables  are  studied,  but  to  learn  how  people  actually  work  and 
play  (Nardi,  1997b).  Many  products  are  designed  and  brought  to  market  with  very  little  idea  of 
how  people  will  use  them  or  whether  they  will  use  them  at  all.  Ethnography  provides  a  basis  on 
which  to  judge  a  product’s  potential  impact  and  can  be  a  fertile  source  of  design  ideas.  The 
leading  theoretical  perspectives  for  ethnographically  oriented  HCI  studies  are  activity  theory, 
distributed  cognition,  and  situated  action.  The  role  of  the  ethnographer  in  design  and  evaluation 
is: 

•  Conducting  specific  studies  for  a  given  project  or  product. 

•  Project  management. 

•  Acting  as  the  “first  user”  of  a  prototype. 

•  Infonning  usability  studies. 

•  Keeping  up  with  the  literature. 

•  Injecting  the  users  perspective  throughout  the  project. 

The  object  of  activity  theory  is  to  understand  the  unity  of  consciousness  and  activity.  It 
incorporates  strong  notions  of  intentionality,  history,  mediation,  collaboration  and  development 
in  constructing  consciousness.  Mediation  is  where  all  human  experience  is  shaped  by  the  tools 
and  sign  systems  we  use.  Ethnographic  and  participatory  design  methods  have  the  problem  that 
every  account  is  an  ad  hoc  description  cast  in  specific  tenns  according  to  the  situation.  Activity 
theory  proposes  that  activity  cannot  be  understood  without  understanding  the  role  of  artifacts  in 
everyday  existence,  especially  the  way  artifacts  are  integrated  into  social  practice.  Activity 
theory  is  concerned  with  practice,  that  is  doing  and  activity,  which  significantly  involves  the 
mastery  of  external  devices  and  tools  of  labor  activity  (Nardi,  1997a). 

Activity  theory  is  both  culturally  given  and  socially  formed  which  makes  it  not  easy  to 
grasp  and  handle  in  empirical  research.  Activity  theory  postulates  that  activity  is  not  simply  a 
prism,  but  rather  it  is  a  prism  that  moves  and  changes  all  the  time  as  a  consequence  of  the 
process  of  learning.  It  is  not  to  be  mistaken  as  development.  Development  is  the  result  of  the 
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learning  that  has  taken  place  because  parts  of  the  world  moved  into  the  scope  of  the  prism  and 
were  reflected  in  and  by  it  (Christiansen,  1997). 

Two  methodologies  used  in  this  kind  of  psychological  research  include  momothetic 
methodology  where  the  data  is  collected  from  a  large  number  of  people  (for  example,  through 
questionnaire  surveys)  and  by  some  process  of  averaging,  purport  to  generalize  with  some  degree 
of  confidence  to  a  larger  population  and  thereby  imply  a  wider  validity.  Idiographic 
methodology  (for  example,  in  depth  interviews)  captures  the  richness  and  complexity  of  the 
phenomenon  under  investigation  but  at  the  risk  of  basing  conclusions  on  a  small  number  of 
potentially  atypical  cases. 

What  distinguishes  ethnographic  research  is  its  purpose,  which  is  cultural  description. 
Ethnographic  research  seeks  to  build  a  systematic  understanding  of  all  human  cultures  from  the 
perspective  of  those  who  have  learned.  This  approach  stresses  things  like  the  actor’s 
understanding  and  theorizing  about  their  actions  as  opposed  to  traditional  social  science  research 
where  the  researcher  attempts  to  explain  human  action  in  tenns  of  psychological  theories  such  as 
attribution  theory.  Ethnography  is  defined  as  the  analytic  descriptions  or  reconstructions  of 
intact  cultural  scenes  and  groups.  Ethnographies  recreate  the  shared  beliefs,  practices,  artifacts, 
folk  knowledge,  and  behaviors  of  some  group  of  people.  Aspects  of  ethnographic  techniques 
include  the  social  situations  or  context  in  which  action  takes  place,  which  is  fundamental  to  the 
analysis  of  the  behaviour. 

Ethnographic  techniques  are  empirical  and  are  almost  without  exception  employed  in 
naturalistic  settings.  The  researcher  is  interested  in  how  individuals  and  groups  behave  in  their 
own  real  world  setting  unmanipulated  by  the  researcher.  This  type  of  research  attempts  to 
present  the  totality  of  the  phenomenon  under  investigation.  Ethnography  makes  extensive  use  of 
qualitative  data  and  has  an  air  of  assisting  in  the  development  and  verification  of  theory  in  order 
to  account  for  human  behaviour.  It  is  concerned  with  revealing  the  routine  and  the  “paramount 
reality”  of  the  everyday  world  of  individuals  and  groups. 

One  touchstone  of  scientific  endeavor  is  described  as  the  replicability  of  the 
investigation.  In  ethnographic  research  it  is  impossible  to  duplicate  naturally  occurring  events  in 
all  their  complexity  and  their  history  because  the  situation  is  constantly  changing,  but  this  does 
not  necessarily  invalidate  the  findings.  One  must  separate  statistical  or  scientific  significance 
from  behavioral  significance.  The  significance  of  an  event  is  independent  of  its  probability  of 
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occurrence.  One  aim  of  ethnographic  research  is  to  record  processes  of  change  as  opposed  to  the 
stability  in  the  environment  (Uzzell,  2000). 

2.3.3.  Human  Computer  Interface. 

Human  computer  interface  (HCI)  is  a  discipline  concerned  with  the  study  and  design  of 
interactive  computing  systems  used  by  people  towards  satisfying  their  goals  (Pirolli,  1999).  It 
consists  of  all  the  objects  and  actions  presented  to  the  user  during  the  process  of  communicating 
with  computer  based  programs  and  applications.  HCI  includes  but  is  not  limited  to  (Keane, 
1992): 

•  The  look  and  feel,  or  style,  of  the  communications  devices.  This  guides  the 
appearance  and  behavior  of  the  interface. 

•  All  of  the  physical  interaction  devices  including  displays,  keyboards,  and  pointer 
devices. 

•  Graphical  interaction  objects  present  on  the  communication  display(s)  such  as 
windows,  icons,  buttons,  and  scroll  bars. 

•  Other  means  of  interaction  such  as  touch  screen  or  voice. 

•  Environmental  factors  such  as  illumination,  seating,  work  place  management, 
keyboard  layout,  display  contrast,  and  symbol  size. 

•  Data  handling  procedures,  data  storage  method,  and  data  processing  logic. 

•  Supporting  hardware  such  as  workstations  and  printers. 

•  The  techniques  employed  by  the  user  to  enter  and  retrieve  infonnation. 

Many  important  elements  of  human  to  human  communication  such  as  age,  sex,  and  race, 
are  totally  absent  from  human  to  computer  interaction  (Williges,  Ehrich,  Williges,  Hartson,  and 
Greenstein,  1984).  The  design  of  any  system  typically  has  a  goal  of  minimizing  both  equipment 
and  personnel  costs.  Typically,  however,  these  two  goals  are  mutually  exclusive.  In  the  past, 
and  in  many  cases  today,  design  tradeoffs  favoring  the  hardware,  primarily  the  computers,  are 
made  at  the  expense  of  the  human  operator.  One  study  suggests  that  high  visual  display  terminal 
(VDT)  work  is  prone  to  the  development,  for  example,  of  stress  related  disorders  in  the  operator 
(Boucsein,  2000).  In  terms  of  human  perfonnance,  however,  the  system  must  attempt  to  achieve 
a  low  error  rate  along  with  an  acceptable  cost  from  the  human  operator  (Williges  and  Williges, 
1981). 
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One  of  humankind’s  oldest  and  most  persistent  dreams  is  to  build  reasoning  devices  of 
which  the  computer  can  be  viewed  as  a  modern  implementation.  However,  in  order  to  be 
effective,  system  designers  must  have  zero  tolerance  for  user-hostile  systems  (Hoffman,  Hayes, 
and  Ford,  2001;  Hoffman,  Klein,  and  Laughery,  2002).  This  philosophy  and  requirement  is  at 
the  heart  of  the  requirement  for  effective  HCI  design. 

Almost  20  years  ago,  as  the  U.S.  Anny  was  seriously  starting  the  development  and 
fielding  of  computer  based  weapon  and  communications  systems,  it  was  recognized  that 
inadequate  design  of  displays  and  input/output  devices  can  degrade  overall  system  perfonnance 
below  the  required  levels  for  functional  effectiveness.  The  problem  is  that,  with  the  advent  of 
computer  based  control  and  communications  devices,  the  human  operator  has  become  the  focal 
point  of  a  potentially  inundating  volume  of  infonnation.  This  is  because  the  computer  can 
acquire,  correlate,  and  present  data  at  a  rate  that  far  exceeds  the  information  processing 
capabilities  of  the  human  operator  (Benel  and  Avery,  1985).  While  one  of  the  fundamental 
reasons  for  introducing  automation  into  complex  system  designs  is  to  reduce  the  probability  of 
human  error  by  reducing  the  anticipated  operator  workload,  in  practice  this  reduction  does  not 
always  occur  (Parasuraman  and  Riley,  1997). 

While  these  comments  were  made  at  a  time  when  many  of  the  C2  and  C3  systems  in  final 
development  and  fielding  today  were  just  in  the  visionary  stage  of  evolution,  many  of  the 
sentiments  expressed  here  and  in  numerous  similar  reports  and  journals,  have  yet  to  find  an 
adequate  solution.  Benel  goes  on  to  state  that  the  C2  operators  must  rely  on  the  system 
designer’s  ability  to  anticipate  their  needs  and  provide  the  information  that  is  needed  when  it  is 
needed  in  order  to  be  able  to  extract  specifically  desired  infonnation  from  the  vast  store  of 
information  contained  within  the  computer  system.  It  is  observed  that  this  is  a  need  that  is  still 
very  much  in  demand  today  and  an  understanding  of  that  need  and  a  solution  for  it  seems  to  still 
elude  C2  system  designers  more  often  than  not. 

2.3.4.  Critical  Incident  Reporting. 

Critical  incident  reporting  is  one  way  to  study  situational  awareness  where  the  critical 
incidents  can  be  presented  in  the  form  of  simulations,  workplace  observations,  or  can  be  elicited 
and  /  or  probed  through  interviews  that  generate  narratives  from  memory  (Klein,  2000a). 

Critical  incident  reports  can  be  a  good  source  of  information  about  how  people  make  use  of 
teamwork  schemas,  especially  in  unusual  situations  where  emerging  conditions  increase  risk  and 
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uncertainty.  Low  schema  similarity  among  team  members  may  be  one  reason  teams  have 
difficulty  addressing  decision  making  requirements  (McNeese  and  Rentsch,  2001). 

The  critical  incident  technique  consists  of  a  set  of  procedures  for  collecting  direct 
observations  of  human  behavior  in  such  a  way  as  to  facilitate  their  potential  usefulness  in  solving 
practical  problems  and  developing  broad  psychological  principles.  An  incident  is  defined  as  any 
observable  human  activity  that  is  sufficiently  complete  in  itself  to  permit  inferences  and 
predictions  to  be  made  about  the  person  perfonning  the  act.  A  critical  incident  is  an  incident  that 
must  occur  in  a  situation  where  the  purpose  or  intent  of  the  act  seems  fairly  clear  to  the  observer 
and  where  its  consequences  are  sufficiently  definite  to  leave  little  doubt  concerning  its  effects. 
Critical  incidents  obtained  from  interviews  can  be  relied  on  to  provide  a  relatively  accurate 
account  of  job  performance  if  suitable  precautions  are  taken  to  prevent  systematic  bias.  The 
essence  of  the  critical  incident  technique  is  that  only  simple  types  of  judgments  are  required  of 
the  observer,  reports  from  only  qualified  observers  are  included,  and  all  observations  are 
evaluated  by  the  observer  in  terms  of  an  agreed  upon  statement  of  the  purpose  of  the  activity. 

The  critical  incident  technique  is  essentially  a  procedure  for  gathering  certain  important 
facts  concerning  behavior  in  defined  situations.  Steps  in  the  critical  incident  technique  include 
classification  of  the  critical  incidents  and  making  inferences  regarding  practical  procedures  for 
improving  performance  based  on  the  observed  incidents.  A  summary  of  the  steps  in  using  the 
critical  incident  activity  (Flanagan,  1954): 

•  Define  the  general  aims  of  the  activity  where  the  job  is  being  perfonned. 

•  Precise  instructions  must  be  given  to  the  observers. 

•  Data  for  behaviors  or  results  observed  should  be  evaluated,  classified,  and 
recorded  while  the  facts  are  still  fresh  in  the  mind  of  the  observer. 

•  Analyze  the  data. 

•  Interpreting  and  Reporting. 

Knowledge  concerning  operator  perfonnance  during  critical  incidents  is  often  obtained 
immediately  following  the  critical  incident  through  personal  interviews  (Randel,  Pugh,  and 
Wyman,  1996).  A  suggested  guide  for  conducting  critical  incident  interviews  is  at  Appendix  D. 

2.3.5.  Task  Analysis. 

Task  Analysis  (TA)  is  a  time  oriented  description  of  personnel,  equipment,  and  software 
interactions  brought  about  by  an  operator,  controller,  or  maintainer  in  accomplishing  a  unit  of 
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work  with  an  item  of  equipment  or  within  a  system.  Critical  tasks  are  those  tasks  involving 
human  perfonnance  that,  if  not  accomplished  in  accordance  with  system  requirements,  will  most 
likely  have  adverse  effects  on  mission  effectiveness,  cost,  system  reliability,  efficiency,  or  safety 
(Myers,  Tijerina,  and  Geddie,  1987).  TA  mainly  focuses  on  an  analysis  of  users’  knowledge, 
preferences,  perceptions,  and  actions,  with  respect  to  the  goal  and  environment  (Pirolli,  1999). 

TA  is  described  as  a  top  down  decomposition  of  the  overall  operation  of  the  system.  The 
steps  in  a  task  analysis  include  developing  a  composite  mission  scenario,  dividing  the  composite 
mission  scenario  into  phases,  identifying  the  segments  in  each  phase,  identifying  the  functions  in 
each  segment  and  identifying  the  tasks  in  each  function  (Hamilton  et  ah,  1991). 

A  task  inventory  taxonomy  is  presented  as  (Myers  et  ah,  1987): 

•  Mission-what  the  system  is  supposed  to  accomplish. 

•  Scenario/conditions-categories  of  factors  for  constraints  under  which  the  system 
will  be  expected  to  operate  and  be  maintained. 

•  Function-categories  of  activity  perfonned  by  a  system. 

•  Job-the  combination  of  all  human  perfonnance  required  for  operation  and 
maintenance  of  one  personnel  position  in  a  system. 

•  Duty-a  set  of  operationally  related  tasks  within  a  job. 

•  Task-A  composite  of  related  activities  performed  for  an  immediate  purpose. 

•  Subtask- Activities  (perceptions,  decisions,  and  responses)  that  fulfill  a  portion  of 
the  immediate  purpose  within  a  task. 

•  Task  element-The  smallest  logically  and  reasonably  definable  unit  of  behavior 
required  in  completing  a  task  or  subtask. 

2.3.6.  Cognitive  Task  Analysis. 

Individual  cognition  is  influenced  by  cognitive  styles  among  individuals.  Cognitive 
styles  are  differences  that  describe  individuals’  preferred  infonnation  gathering  and  decision¬ 
making  abilities.  Personality  research  has  indicated  the  presence  of  five  recurring  personality 
dimensions  (Costa  and  McCrae,  1995;  Thompson,  1998): 

o  Extroversion-interpersonally  based  traits  such  as  sociability,  assertiveness, 
dominance,  and  the  tendency  to  be  outgoing  versus  reserved,  aloof,  shy, 
and  solemn. 

o  Agreeableness-interpersonal  in  nature  and  includes  tendencies  to  be 
tolerant,  cooperative,  and  wann  versus  malicious,  harsh,  irritable,  and 
insincere. 
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o  Conscientiousness:  includes  thoroughness,  persistence,  predictability, 
rigidity  and  dependability  versus  carelessness,  absent-mindedness, 
forgetful  and  erratic. 

o  Neuroticism:  one’s  emotional  resilience,  calmness,  stability,  confidence, 
and  independence  versus  a  tendency  to  be  anxious,  fearful,  sensitive,  and 
self-critical. 

o  Openness  to  Experience-includes  tendencies  to  be  intellectually  complex, 
insightful,  original,  curious,  and  studious  versus  dull,  illogical  and  narrow 
minded. 

Much  of  modem  cognitive  science  has  been  devoted  to  providing  detailed  models  of 
cognitive  operations  and  developing  experimental  methods  and  paradigms  to  infer  these 
processes  (Williges,  1987).  However,  cognitive  processes  such  as  decision  time,  accuracy  of 
answers,  good  vs.  bad  decisions,  etc.,  cannot  be  casually  observed  (Braun  et  al.,  1999). 

Methodologies  that  can  be  tools  for  identification  of  cognitive  requirements  early  in  the 
design  cycle  include  Cognitive  Task  Analysis  (CTA),  Scenarios  and  Team  Integrated  Design 
Environment  (TIDE).  CTA  comprises  both  knowledge  elicitation  (interviews  and  observations 
with  SMEs)  and  knowledge  representation  (analysis  and  meaningful  representation  of  the  data) 
(McDermott,  Klein,  Thordsen,  Ransom,  and  Paley,  2000).  CTA  is  the  description  of  the 
expertise  needed  to  perform  complex  tasks  (Klein,  1998).  The  difference  in  task  analysis  (TA) 
and  CTA  is  that  TA  focuses  only  on  observable  behavior  with  the  result  that  there  is  no 
information  gained  about  the  overall  organization  of  knowledge.  CTA  focuses  on  the 
psychological  processes  underlying  the  behavior  and  concentrates  on  the  critical  decisions  and 
cognitive  processes  that  separate  the  expert  from  the  novice  (Brenner,  Sheehan,  Arthur,  and 
Bennett,  1999).  In  essence,  it  analyzes  the  thought  processes  of  performers  while  they  complete 
a  task  (Randel  et  al.,  1996).  CTA  is  a  method  for  capturing  expertise  and  making  it  accessible 
for  training  and  system  design  (Klein,  1998).  CTA  goes  beyond  traditional  task  analysis  that 
have  concentrated  on  the  procedures  to  be  followed  and  have  had  relatively  little  to  say  about 
perception,  judgment,  and  decision  making  skills.  The  steps  in  the  process  are: 

•  Locate  sources  of  expertise  (and  acquire  background  knowledge  in  the  process). 

•  Evaluate  the  quality  of  the  expertise. 

•  Perform  knowledge  elicitation  to  get  inside  the  head  of  the  skilled  decision 
makers. 

•  Process  the  findings  so  they  can  be  interpreted  to  others. 
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•  Apply  the  findings. 

Key  points  of  CTA  are  identified  as: 

•  Experts  can  perceive  things  that  are  invisible  to  novices. 

•  Skilled  chess  players  show  high  quality  moves,  even  under  extreme  time  pressure, 
and  high  quality  moves  as  the  first  ones  they  consider. 

•  Training  to  high  skill  levels  should  emphasize  perceptual  skills,  along  with 
mastery  of  procedures. 

CTA  can  be  a  tool  for  identification  of  cognitive  requirements  early  in  the  design  cycle. 

It  comprises  both  knowledge  elicitation  (interviews  and  observations  with  SMEs)  and  knowledge 
representation  (analysis  and  meaningful  representation  of  the  data)  (McDermott  et  al.,  2000). 

The  purpose  of  CTA  techniques  is  to  analyze  and  model  the  cognitive  processes  that  give  rise  to 
human  task  perfonnance  in  specific  domains,  as  the  basis  for  design  and  evaluation  of  computer- 
based  systems  and  their  user  interfaces  (Zachary  et  al.,  1998). 

2.4.  Simulation  Modeling  Concepts  and  Procedures. 

Modern  computer  simulation  software  architectures  and  hardware  platforms  have  made 
possible  a  level  of  modeling  sophistication  that  simply  did  not  exist  as  recently  as  20  years  ago. 
Conventional  linear  programming  techniques  supplied  by  languages  such  as  Fortran  and  C  have 
been  supplanted  by  discrete  event  language  capabilities  that  allow  model  developers  and 
computer  programmers  to  focus  more  on  the  design  of  the  software  rather  than  the  details  of  the 
coding  language.  This  has  greatly  enhanced  the  ability  to  translate  conceptual  models  into 
working  computer  simulations  that  are  not  only  easier  to  code  into  computer  programs,  but  also 
provide  a  more  accurate  and  realistic  representation  of  the  environment  being  simulated. 

This  research  uses  two  of  these  modern  simulation  approaches  in  the  development  of  the 
COMPASS  paradigm.  These  are  neural  network  simulations  and  discrete  event  simulations. 
Some  of  the  underlying  principles  behind  these  modeling  approaches  along  with  certain  concepts 
pertinent  to  an  understanding  of  computer  simulation  based  investigative  approaches  are 
presented  here  to  facilitate  an  understanding  of  this  process. 

2.4.1.  The  System. 

A  system  is  an  assemblage  or  combination  of  things  or  parts  forming  a  complex  or 
unitary  whole  (Hiemstra  et  al.,  1987).  Systems  typically  include  various  kinds  of  real-world 


99 


facilities  or  processes.  At  its  top  level  it  is  an  entity  that  exists  to  carry  out  some  purpose  and  is 
typically  composed  of  some  combination  of  humans,  machines,  and  other  things  such  as 
environmental  factors.  It  exists  to  carry  out  some  goal  that  cannot  be  accomplished  by  the 
individual  components  working  independently.  A  “human-machine”  system  is  a  combination  of 
one  or  more  humans  and  one  or  more  physical  components  interacting  to  bring  about  some 
desired  output  resulting  from  given  inputs  (Sanders  and  McCormick,  1993).  It  is  defined  in 
more  detail  as  a  group  of  objects  that  are  joined  together  with  some  form  of  interaction  or 
interdependence  for  the  purpose  of  accomplishing  some  goal  or  goals  (Banks,  Carson,  and 
Nelson,  1996).  It  is  a  collection  of  entities  such  as  people  and/or  machines  that  act  and  interact 
together  to  accomplish  a  goal  (Law  and  Kelton,  2000).  In  more  general  terms,  a  system  is  an 
aggregation  of  elements  according  to  some  structure  to  accomplish  system  goals  and  objectives. 
All  systems  include  the  following  characteristics: 

•  Interaction  of  elements. 

•  Structure. 

•  Purpose  and  goals. 

•  Inputs. 

•  Outputs. 

Systems  are  usually  composed  of  some  combination  of  humans  and  machines.  They  have  a 
defined  structure  and  organizations.  Systems  are  further  characterized  as  having  external 
boundaries  that  separate  them  from  elements  outside  of  the  system  (Czaja,  1997).  The  systems 
approach  considers  total  system  perfonnance  rather  than  concentrating  on  individual  parts.  This 
concept  is  based  on  the  understanding  that  even  if  the  various  subparts  of  a  system  are  optimized 
for  performance,  the  performance  of  the  overall  system  may  be  sub  optimal  (Pegden,  Shannon, 
and  Sadowski,  1995).  The  field  of  macroergonomics  provides  the  concept  of  “work  system” 
which  involves  two  or  more  people  interacting  with  the  environment.  This  environment  may 
include  hardware  and/or  software,  external  environment,  internal  environment,  or  an 
organizational  design  (Hendrick  and  Kleiner,  2001).  For  the  purposes  of  this  research  the  tenn 
system  is  defined  as  including  the  weapon  systems  and  all  of  the  people  and  equipment 
necessary  to  field  and  sustain  these  weapon  systems  in  peacetime  and  combat.  Figure  16  shows 
the  components  of  a  basic  system. 
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Figure  16  -  A  Simple  System 
(Littlefield,  1998) 


2.4.2.  Computer  Simulation  Models. 

Most  computer  simulations  used  in  human  computer  interface  design  are  task  network 
models  which  structure  the  interface  around  the  task,  subtasks,  interconnections  of  subtasks, 
rules  for  connecting  subtasks  and  time  to  complete  the  subtasks  (Williges,  1987).  One 
simulation  language  in  particular,  Micro  Saint™  has  been  specifically  designed  for  modeling 
systems  where  the  human  is  a  part  of  the  system  (Laughery,  1989b;  Laughery,  1999;  Laughery 
and  Corker,  1997;  Williges,  1987).  Williges  goes  on  to  point  out  that  no  single  complete  model 
of  the  human  computer  interface  exists.  This  statement  seems  just  as  valid  today  as  it  was  then 
and  will  likely  remain  so  for  the  indefinite  future.  He  goes  on  to  point  out  that  the  challenge  for 
user  modeling  is  to  develop  models  that  can  be  readily  used  by  designers.  Table  7  shows  a 
summary  of  Williges  recommended  modeling  approaches  to  human-computer  systems. 

Table  7  -  Summary  of  Human  System  Modeling  Approaches 

(Williges,  1987) 


Type  o  f  Model 

Emphasis  of  Model 

Design  Tools 

Design  Stage 

1.  Conceptual 

Cognitive  processes 

Procedural  representation 

User  actions 

Tiror  analysis 

Verbal  protocols 

Initial  design 

Formative  evaluation 

Cognitive  structure 

Task  representation 

Device  representation 

Production  rules 

Transition  networks 

Initial  design 

Cognitive  strategy 

Task  representation 

Goal  representation 

Task  analysis 

Goal  analysis 

Initial  design 

2.  Quantitative 

Performance 

User  performance 
prediction 

Keystroke  analysis 

Initial  design 

Formative  evaluation 
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Type  o  f  Model 

Emphasis  of  Model 

Design  Tools 

Design  Stage 

Procedural  representation 

Ergonomic 

Anthropometric 

representation 

Biomechanical 

representation 

Computer  aided  design 

Initial  design 

Computer  simulation 

Task  sequence 

representation 

Simulation  languages 

Initial  design 

Statistical 

Task  representation 

User  performance 
prediction 

Clustering  algorithm 
Polynomial  regression 

Initial  design 

Formative  evaluation 

2.4.3.  Task  Network  Modeling. 

Network  models  in  general  evolved  from  the  difficulty  to  develop  fonnal  mathematical 
models  of  crew  performance.  This  occurred  because  theoretical  constructs  were  not  available  to 
develop  equations  that  could  describe  the  perfonnance  of  multi-operator  systems.  While 
network  models  are  often  criticized  as  to  their  validity  and  because  they  are  often  viewed  as 
being  too  general,  there  is  an  opinion  that  many  times  they  are  the  only  alternative  for 
systematically  evaluating  alternative  system  designs  (McMillan  et  al.,  1991).  Network  models 
based  on  task  performance,  i.e.,  task  network  modeling,  can  provide  a  sound  method  for 
modeling  human  behaviors  or  performance  in  systems  (Laughery,  1989b).  Human  behavior  is 
defined  as  any  human  action  generally  defined  by  a  stimulus  or  cue  and  a  response.  A  basic 
stimulus-organism-response  constituent  of  behavior  is  comprised  of  the  smallest  logically 
definable  set  of  perceptions,  decisions,  and  responses  required  to  complete  a  task  (Godowski, 
King,  Ronco,  and  Askren,  1978). 

Approaches  in  task  network  model  are  based  on  the  concepts  of  task  allocation  and 
workload  prediction.  These  are  tools  that  address  the  problem  of  allocating  functions  to  the 
human  or  to  the  machine.  Some  of  the  approaches  to  task  allocation  and  workload  prediction 
include  time  line  analysis  which  compares  the  time  required  to  complete  assigned  tasks  to  the 
time  available  and  multiple  resource  or  attention  demand  which  recognizes  the  fact  that  some 
tasks  can  be  accomplished  essentially  in  parallel  (low  attention  demand),  while  other  tasks  must 
be  done  serially  (high  attention  demand). 

The  steps  that  are  followed  in  task  network  modeling  start  with  the  identification  of  the 
predecessor  tasks  that  must  be  completed  before  the  task  in  question  can  begin.  Then  the 
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statistical  characteristics  of  the  task  in  question  are  developed.  Finally,  the  branching  to  other 
tasks  to  be  performed  upon  task  completion  is  identified  (McMillan  and  Martin,  1989). 

Current  thinking  classifies  the  world  of  human  -  system  performance  models  into 
two  categories.  The  first  is  tenned  reductionist  models.  These  use  human  system  task 
sequences  as  the  primary  organizing  structure  of  the  model.  The  term  reductionist  is  used 
to  describe  the  process  of  taking  larger  aspects  of  human  system  behavior  and 
successively  reducing  it  to  smaller  elements  of  behavior  until  reasonable  estimates  of 
human  perfonnance  for  the  task  elements  can  be  made.  Task  networking  models  are  an 
example  of  reductionist  models  where  the  basis  of  a  representation  of  human 
performance  is  the  task  analysis  (Laughery  and  Corker,  1997). 

In  task  network  models  human  performance  is  decomposed  into  a  series  of  sub 
functions  that  are  further  decomposed  into  tasks.  The  sequence  of  tasks  to  be  performed 
to  replicate  the  system  according  to  a  task  analysis  is  what  constitutes  the  task  network. 

The  term  task  is  defined  as  a  composite  of  related  activities  (behaviors)  performed  by  an 
individual  and  directed  toward  accomplishing  a  specific  amount  of  work  within  a  specific 
work  context  (Godowski  et  ah,  1978).  Each  task  has  a  goal.  Another  definition  is  the 
simplest  level  of  behavior  that  describes  the  perfonnance  of  a  meaningful  function  in  a 
job  under  consideration.  Task  analysis  is  defined  as  a  process  of  reviewing  actual  job 
content  and  context  in  order  to  classify  information  into  units  of  work  within  a  job 
(Hiemstra  et  ah,  1987).  Another  definition  is  that  is  an  analytic  process  employed  to 
determine  the  specific  behaviors  required  of  a  human  component  in  a  man-machine 
system  (Godowski  et  ah,  1978).  This  form  of  modeling  has  become  popular  for  several 
reasons.  First,  it  is  a  direct  means  of  applying  conventional  task  analysis  procedures  into 
a  simulation-modeling  environment.  This  modeling  system  also  typically  provides  the 
ability  to  establish  sub-task  networks  of  detailed  systems  and  established  closed  loop 
approximations  of  the  system  performance.  Next,  recent  advances  in  technology  have 
made  task  network  modeling  relatively  easy  to  understand  and  to  implement  and  has 
made  it  accessible  to  a  wide  audience.  Finally,  task  network  modeling  can  address  many 
different  types  of  issues  and  allow  many  different  types  of  systems  to  be  described  and 
evaluated  (Laughery  and  Corker,  1997).  An  example  of  a  programming  environment  that 
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implements  this  type  of  modeling  tool  is  the  Micro  Saint™  discrete  event  programming 
language. 

A  second  major  category  of  models  of  human  behavior  is  tenned  the  First 
Principle  models,  which  are  structured  around  an  organizing  framework  that  represents 
the  goals  and  constructs  of  the  human  perfonnance  system  being  modeled.  The  term, 
first  principle,  is  based  on  the  fundamental  principles  of  humans  and  their  interactions 
with  the  system  and  the  environment.  These  models  have  structures  embedded  into  them 
that  directly  represent  elemental  components  of  human  performance.  Examples  of  these 
behavioral  structures  might  include  goal  seeking,  sensation,  perception,  cognition,  and 
motor  output.  First  Principle  models  require  descriptions  of  how  the  system  and 
environment  interacts  with  the  human  process  being  modeled.  An  example  of  this  type 
of  modeling  is  the  Man  Machine  Integrated  Design  and  Analysis  System  (MIDAS) 
(Laughery  and  Corker,  1997). 

When  using  task  network  modeling  to  describe  human  system  performance  where 
basic  human  behavior  characteristics  are  lacking,  one  approach  to  overcome  the 
limitations  of  the  task  network  model  is  to  augment  it  with  elements  of  First  Principle 
models  to  represent  behavioral  phenomena  accurately  (Laughery  and  Corker,  1997).  The 
advantages  of  this  approach  would  be  to  capitalize  on  the  strengths  of  each  modeling  type 
to  provide  a  strong  representation  of  human  system  perfonnance  that  is  moderated  with 
human  behavior  traits.  The  disadvantage  of  this  approach,  as  evidenced  that  there  is  no 
example  in  the  literature  reflecting  it,  is  that  integrating  two  totally  different  modeling 
structures,  each  with  their  own  database  structures  and  simulation  mechanisms,  is 
apparently  much  more  difficult  to  perfonn  than  it  is  to  state. 

Other  approaches  to  modeling  human  perfonnance,  especially  in  military 
command  and  control  as  well  as  conventional  managerial  situations,  includes  classical 
linear  programming  and  descriptive  programming  techniques  that  are  provided  by 
conventional  programming  languages  such  as  Fortran  and  C++.  It  also  includes  the 
whole  general  category  of  mathematical  modeling. 

Mathematical  modeling  has  strong  application  in  the  physical  sciences  arena 
because  of  its  ability  to  provide  a  continuous  description  of  the  physical  system  over 
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time.  The  “only”  requirement  is  that  the  system  under  investigation  must  be  able  to  be 
described  in  tenns  of  mathematical  functions  and  equations.  Chemical  reactions,  nuclear 
explosions,  even  weather  patterns  are  examples  of  physical  phenomena  that  have  been 
modeled  in  this  way.  However,  the  inexact  science  of  trying  to  estimate  human 
performance  characteristics  has,  so  far,  eluded  acceptable  mathematical  models  that 
describe  it.  There  are  attempts  to  do  so  where  the  model  is  augmented  by  Monte  Carlo 
randomness  to  account  for  human  variability. 

These  types  of  models  generally  follow  the  predictive  equations  to  predict  force 
on  force  attrition  (Lanchester,  1916)  developed  early  in  the  20th  century.  Because  of  the 
random  nature  of  human  variability  these  models  typically  have  found  application  in  the 
military  training  arena  as  command  and  control  training  platforms  (Middlebrooks,  1991). 
They  are  typically  implemented  in  a  conventional  linear  programming  environment  with 
languages  such  as  Fortran  and  C++  and  require  the  model  developer  to  account  for  all 
aspects  of  the  system  being  modeled  as  the  environment  only  provides  technical  tools  to 
support  the  programming  constructs.  Thus,  all  algorithms,  interactions,  interaction 
techniques,  and  model  constructs  must  be  provided  by  the  model  developer  in  order  to 
describe  the  system  being  modeled. 

There  are  other  methods  for  modeling  decision  making  in  command  and  control 
that  include  human  participation,  game  theory,  optimization,  mechanical  statistical 
techniques,  controlled  experimentation  and  expert  systems  (Farrell,  Bonder,  Proegler, 
Miller,  and  Thompson,  1986).  However,  most  of  these  predate  modem  computer 
simulation  capabilities.  Human  participation  translates  to  full-scale  field  exercises,  which 
almost  always  had  to  be  scripted  with  predetennined  outcomes  because  of  peacetime 
maneuver  limitations.  Game  theory  models  usually  were  pre-computer  board  games  with 
outcomes  determined  many  times  by  actual  rolls  of  the  dice.  Optimization  modeling 
techniques  using  mathematical  equation  descriptors  still  find  favor  for  force  on  force 
evaluations  but  lack  the  component  of  the  human  in  the  system  as  already  described. 
Mechanical  statistical  techniques  follow  the  limitations  of  other  statistical  models  already 
discussed.  Controlled  experimentation  would  be  a  logical  first  choice  for  command  and 
control  evaluations  except  for  the  fact  that  qualified  test  participants  are  almost  non¬ 
existent  for  command  and  control  evaluations.  To  run  a  controlled  experiment  evaluating 


105 


battalion  level  command  and  control  requires  the  availability  of  at  least  one,  and  probably 
several,  battalion  command  and  control  team(s)  of  approximately  23  people  for  the 
duration  of  the  experiment.  However,  there  are  instances  where  this  has  been  attempted 
such  as  a  study  conducted  at  Fort  Benning,  Georgia  in  the  early  1970s  where  10  -  12  man 
teams  of  combat  experienced  officers  were  brought  together  in  an  ad  hoc  fashion  over  a 
three  week  period  to  role  play  TOC  operations  under  experimentally  controlled  scenarios 
(Olmstead  et  ah,  1973).  The  drawback  to  this  type  of  study  is  that  there  is  no  team 
cohesion  or  familiarity  among  the  team  members  because  they  were  nominally  brought 
together  just  for  the  study  and  had  no  part  in  the  preparation  phase  of  the  tactical 
operation  as  would  have  been  the  case  in  a  real  battalion  operation. 

The  only  place  to  obtain  qualified  C2  teams  is  from  actual  military  battalions, 
which  is  generally  precluded  by  their  own  requirements  for  training  and  mission  support. 
Expert  systems,  after  a  brief  promise  of  great  expectations  in  the  middle  1980s,  have 
generally  failed  to  live  up  to  their  billing  primarily  because  of  the  same  problems  with  the 
mathematical  models,  which  is  the  unavailability  of  suitable  and  accurate  algorithmic 
predictors  of  human  performance  under  varying  conditions. 

2.4.4.  Optimization  Techniques  Using  Computer  Simulation  Models. 

Computer  simulation  is  defined  as  the  establishment  of  a  mathematical-logical 
model  of  a  system  and  the  experimental  manipulation  of  that  model  on  a  digital  computer 
(Biles  and  Ozmen,  1987;  Biles  and  Swain,  1979).  Four  major  classes  of  approaches  to 
simulation  optimization  are  described  as  (Pierreval,  Tautou,  and  Bzeznik,  1995): 

-1 .  Gradient  based  search  methods. 

-2.  Stochastic  approximation  methods. 

-3.  Response  surface  methodology. 

-4.  Heuristic  methods. 

The  aim  of  each  of  these  approaches  is  to  propose  a  strategy  to  explore  the  solution  space 
with  a  limited  number  of  simulation  experiments.  From  these  approaches  there  are  three 
principal  experimental  design  techniques  for  statistical  analysis  and  optimization  using 
computer  simulation,  especially  in  a  multi-computing  environment.  These  are  factor 
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screening  experiments,  experiments  of  comparison,  and  response  surface  methodology 
(Biles  and  Ozmen,  1987).  Another  view  of  optimizing  the  output  proposes  the  steps 
(Clayton,  Weber,  and  Taylor,  1982): 

•  Enumerate  all  possible  combinations  of  inputs  in  the  model  and  evaluate  the 
output  for  each  combination.  This  is  feasible  only  for  models  with  a  few  inputs 
and  short  computer  simulation  time. 

•  Run  various  combinations  of  input  variable  values  picked  from  an  experimental 
design  analysis.  Then,  using  regression  analysis,  estimate  the  equations  that 
produce  the  output  values. 

•  Use  a  direct  search  procedure  that  considers  multi-objectives  and  does  not  require 
knowledge  of  the  exact  model  equations.  The  simulation  model  is  treated  as  a 
“black  box”  in  which  the  inputs  are  varied  according  to  the  search  procedure. 

This  makes  decisions  on  the  next  set  of  inputs  to  try  based  on  the  observed  model 
outputs  from  the  current  inputs. 

Another  categorization  states  that  optimization  procedures  fall  into  the  three 
categories  of  (Biles  and  Swain,  1979): 

•  Direct  search  techniques. 

•  First  order  response  surface  methods. 

•  Second  order  response  surface  procedures. 

The  term  design  of  experiments  (DOE),  which  is  related  to  the  term  experimental 
design,  is  a  subdiscipline  within  mathematical  statistics.  While  classical  experimental 
design  techniques  can  allow  simulation  outputs  to  be  approximated  by  polynomials  that 
are  functions  of  input  parameters,  DOE  techniques  can  be  used  to  conduct,  for  example, 
2s  -  2  level  full  factorial  experiments.  This  design  would  involve  256  treatment 
combinations,  or  more,  which  is  not  normally  associated  as  being  possible  in 
conventional  experimental  design  (Pucik,  Curry,  Dziuban,  and  Senseny,  1999).  DOE 
includes  designs  such  as  2k'p  (fractional  factorial  experiments)  as  they  are  applied  to 
computer  simulation.  DOE  gives  good  estimators  of  the  main  effects,  interactions,  and 
quadratic  effects  through  regression  metamodels.  Thus,  DOE  improves  the  effectiveness 


107 


of  simulation  experimentation.  DOE  requires  relatively  few  simulation  runs  resulting  in 
improved  efficiency  of  both  the  experimental  design  and  the  use  of  computer  resources. 
Once  the  factor  effects  have  been  quantified  through  regression  estimates  they  can  be 
used  for  optimization  through  response  surface  modeling  which  augments  regression 
analysis  and  DOE  through  the  use  of  techniques  such  as  the  steepest  ascent  hill  climbing 
technique  (Kleijnen,  1998). 

Some  of  the  peculiarities  of  computer  simulation  include,  first,  the  fact  that  there 
can  be  a  great  many  factors  in  practical  simulation  models.  As  many  as  281  factors 
would  not  be  unusual  while  standard  experimental  design  would  typically  not  have  more 
than  15,  if  that  many.  Stochastic  simulation  models  use  pseudo  random  numbers,  which 
gives  the  analysts  more  control  over  the  noise  in  their  experiments  than  investigators  in 
standard  statistical  experiments.  Both  common  and  antithetic  seeds  may  be  used. 
(Anththetic  random  numbers  are  used  to  induce  a  negative  covariance  between  paired 
replications.  If  a  random  number  in  a  replication  is  r,  derived  as  a  uniform  random 
number  in  the  range  from  0  to  1,  i.e.,  /}■  ~  UNIF(0,1),  then  the  corresponding  antithetic 
random  number  is  1  -ij.  Antithetic  random  number  seeds  are  detennined  using  the  same 
inverse  relationship  (Pegden  et  ah,  1995)  (Law  and  Kelton,  2000)  ). 

In  non- simulation  experiments  randomization  is  a  major  concern  where  the 
experimental  units  are  assigned  in  a  random  nonsystematic  way  in  order  to  avoid  bias. 
For  simulation  based  DOE  this  concern  disappears  as  pseudorandom  number  streams 
take  over.  Finally,  blocking  can  be  an  important  technique  in  conventional 
experimentation  to  reduce  systematic  differences  among  experimental  units.  DOE 
simulations  exercise  complete  blocking  control  over  the  experiment  which  eliminates  the 
need  for  blocking  although  it  may  be  used  to  assign  common  and  antithetic  pseudo 
random  numbers  (Kleijnen,  1998). 

Terminology  associated  with  DOE  includes  (Kleijnen,  1998): 

•  Factor  -  a  parameter,  input  variable,  or  a  module  of  a  simulation  model  or 
simulation  computer  program.  The  factor  may  be  qualitative. 

28 1 

•  Number  of  combinations  of  factor  levels  -  2“  ,  this  would  not  be  unusual 
in  computer  simulations. 
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•  What  if  analysis  -  DOE  analyzes  the  input  /  output  (I/O)  data  of  the 
experiment  or  simulation  to  derive  conclusions  about  the  importance  of 
the  factors. 

•  Sensitivity  analysis  -  the  systematic  investigation  of  the  reaction  of  the 
simulation  responses  to  extreme  values  of  the  model’s  input  or  to  drastic 
changes  in  the  model’s  structure. 

•  Regression  analysis  -  (also  known  as  analysis  of  variance  -  ANOVA)  is 
based  on  a  metamodel,  which  is  defined  as  a  model  of  the  underlying 
simulation  model.  A  metamodel  is  an  approximation  of  the  simulation 
program’s  I/O  transfonnation  and  is  also  called  a  response  surface. 

•  Responses  -  simulation  outputs.  Most  simulations  have  multiple 
responses. 

DOE  Advantages: 

•  DOE  can  be  applied  to  all  simulation  models,  either  deterministic  or 
stochastic. 

•  DOE  provides  better  estimates  of  the  factor  effects  than  does  the  one 
factor  at  a  time  approach. 

•  DOE  may  be  used  not  only  for  sensitivity  analysis  and  optimization,  but 
also  for  validation. 

DOE  Disadvantages: 

•  DOE  cannot  take  advantage  of  the  specific  structure  of  a  given  simulation 
which  requires  more  simulation  runs  for  the  analysis  than  do  perturbation 
analysis  and  modern  importance  sampling  which  is  also  known  as 
likelihood  ratios  or  score  functions. 

One  purpose  of  DOE  is  the  optimization  of  the  simulated  system.  Some  examples 
of  optimization  techniques  include  sequential  simplex  search,  simulated  annealing  that  is 
a  network  flow  method  based  on  a  physical  process  in  metallurgy,  so  called  genetic 
methods  that  seek  to  imitate  the  biological  phenomenon  of  evolutionary  reproduction  and 
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tabu  search  that  may  be  regarded  as  a  technique  based  of  selected  concepts  from  artificial 
intelligence  that  is  a  general  heuristic  procedure  for  guiding  search  to  obtain  good 
solutions  in  complex  solution  spaces  (Glover,  Taillard,  and  de  Werra,  1993). 

Commercial  software  is  available  for  simulation  modeling  using  some  of  these 
techniques.  Some  of  these  packages  include  ProModel’s  SimRunner,  Witness’ 

Optimizer,  and  Micro  Saint™’s  OptQuest.  All  of  these  programs  can  handle  single 
response  univariate  cases,  however,  as  of  1996  there  was  no  known  software  for 
optimization  of  multiple  responses  in  the  multivariate  case  (Kleijnen,  1998). 

The  use  of  parallel-distributed  processors  (PDP)  and/or  multi-computers  is  often 
employed  in  computer  simulation  because  of  the  complex  and  extensive  computational 
loading  that  these  simulations  can  place  on  the  computer.  A  multi-computer  is  defined  as 
a  set  of  tightly  coupled  but  autonomous  computers,  capable  of  synchronizing  and 
communicating  in  parallel  while  operating  independently.  They  often  employ  shared 
resources  such  as  memory  and  input-output  devices.  PDP  networks,  on  the  other  hand 
involve  groupings  of  completely  independent  computer  systems  that  may  or  may  not  be 
collocated  that  work  together  to  solve  complex  problems  and  simulations  and  typically 
are  coordinated  in  their  efforts  through  a  central  executive  computer  system  that  provides 
directives  and  receives  responses  back  from  the  individual  processors  on  the  network.  A 
distributed  simulation  is  defined  as  the  process  by  which  large,  complex  simulation 
models  are  decomposed  onto  a  set  of  processors.  Two  steps  involved  in  this  are  event 
decomposition  in  which  each  of  the  several  events  making  up  a  model  is  assigned  to  a 
specific  processor  and  task  decomposition  in  which  the  various  simulation  functions  such 
as  I/O,  event  processing,  random  number  and  random  variates  generation,  statistics 
collection,  and  report  generation  are  allocated  to  different  processors  (Biles  and  Ozmen, 
1987). 

One  multi-computer  approach  uses  a  JAVA  based  system  for  allocating 
simulation  trials  to  a  set  of  P  parallel  processors  for  carrying  out  simulations  involving 
direct  search  optimization  or  response  surface  methodology  procedures.  As  opposed  to  a 
PDP  based  simulation  where  the  simulation  model  is  decomposed  and  its  parts  run  in  a 
parallel  environment,  the  parallel  replications  approach  allows  a  simulation  model  to  run 
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to  completion  with  a  unique  set  of  input  conditions.  Since  a  simulation  study  typically 
involves  executing  R  replications  of  the  model  at  each  of  S  sets  of  input  conditions,  the 
server’s  task  in  managing  a  parallel  replications  approach  is  to  allocate  the  RS  x 
simulation  trials  to  P  client  processors  in  a  manner  that  balances  the  workload  on  each 
processor.  The  objective  is  to  complete  the  simulation  study  in  a  time  interval 
approaching  1/P  of  that  which  would  be  required  of  a  single  processor  operating  in  a 
purely  sequential  mode  (Biles  and  Kleijnen,  1999). 

A  sampling  follows  of  some  of  these  techniques  from  the  literature  for  optimizing 
the  results  of  computer  simulation  models. 

2.4. 4.1.  Tabu  Search. 

TABU  Search  (TS)  is  a  heuristic  for  providing  solutions  to  combinatorial  problems  by 
moving  from  one  solution  to  another  in  a  way  that  avoids  becoming  trapped  in  local  optimal 
solutions.  TS  records  the  best  solutions  discovered  during  the  search  and,  often,  these  solutions 
are  optimal.  TS,  however,  does  not  guarantee  finding  an  optimal  solution,  nor  will  it  recognize 
an  optimal  solution  if  it  encounters  one  (Ryan,  Bailey,  Moore,  and  Carlton,  1998).  One  of  the 
main  components  of  TS  is  its  use  of  flexible  (adaptive)  memory,  which  plays  an  essential  role  in 
the  search  process.  The  method  behind  TS  can  be  viewed  as  an  iterative  technique  which 
explores  a  set  of  problem  solutions,  denoted  by  A,  by  repeatedly  making  moves  from  one 
solution  s  to  another  solution  s  ’  located  in  the  neighborhood  N(s)  of  s.  These  moves  are 
performed  with  the  aim  of  efficiently  reaching  a  solution  that  qualifies  as  “good”  (optimal  or 
near  optimal)  by  the  evaluation  of  some  objective  function  to  be  minimized.  The  goal  is  to  make 
improving  moves  to  the  fullest  extent  allowed  while  balancing  trade-offs  between  solution 
quality  and  computational  effort  in  examining  larger  samples.  TS  may  be  viewed  as  a  variable 
neighborhood  method  where  each  step  redefines  the  neighborhood  from  which  the  next  solution 
will  be  drawn,  based  on  the  conditions  that  classify  certain  moves  as  tabu.  Experience  has 
shown  that  TS  is  able  to  obtain  results  that  match  or  surpass  the  best  known  outcomes  from  other 
techniques  in  a  variety  of  optimization  settings.  One  advantage  of  TS  is  its  ability  to  adapt  a 
rudimentary  prototype  implementation  to  encompass  additional  model  elements  such  as  new 
types  of  constraints  and  objective  functions  (Glover  et  ah,  1993). 
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2.4A.2.  Goal  Programming  With  Preemptive  Priorities. 

Many  times  the  simulation  analyst  is  not  interested  in  the  detailed  specifications  of  the 
model,  they  only  want  to  know  what  combination  of  input  variables  will  provide  the  optimal 
output.  Some  of  the  possible  approaches  to  optimizing  the  output  include  the  enumeration  of  all 
possible  combinations  of  inputs  in  the  model  and  evaluate  the  output  for  each  combination. 
However,  this  is  feasible  only  for  models  with  a  few  inputs  and  short  computer  simulation 
execution  times.  A  second  approach  is  to  run  various  combinations  of  input  variable  values 
picked  from  an  experimental  design  analysis.  Then,  using  regression  analysis,  estimate  the 
equations  that  produce  the  output  values  and  apply  math  programming  optimization  techniques 
to  the  estimated  model.  This,  however,  may  result  in  an  estimated  model  that  is  nonlinear  and 
requires  multiple  objectives.  A  third  approach  is  to  use  a  direct  search  procedure  that  considers 
multiple  objectives  and  does  not  require  knowledge  of  the  exact  model  equations.  The 
simulation  model  is  treated  as  a  “black  box”  in  which  the  inputs  are  varied  according  to  the 
search  procedure.  This  makes  decisions  on  the  next  set  of  inputs  to  try  based  on  the  observed 
model  outputs  from  the  current  inputs.  This  third  technique  is  a  method  for  optimizing  multiple 
response  simulation  models  that  applies  modified  pattern  search  and  gradient  search  techniques 
to  the  simulation  model  responses  that  may  be  linear  or  nonlinear.  For  this  procedure  the  model 
equations  generating  the  responses  may  be  known  or  unknown,  however,  the  simulation  must  not 
contain  Monte-Carlo  random  effects  (Clayton  et  al.,  1982). 

This  technique  is  a  procedure  for  solving  integer  nonlinear  optimization  problems.  The 
process  consists  of  a  ten-step  procedure,  which  is  summarized  as: 

-Step  1 :  Choose  an  initial  starting  point  and  evaluate  the  goal  values  associated  with  it. 

-Step  2:  Attempt  to  find  a  direction  vector  using  the  pattern  search  exploratory  move 

method. 

-Step  3:  Check  to  detennine  whether  or  not  the  direction  vector  from  step  2  is  equal  to 
zero  and  then  send  the  program  in  the  proper  direction. 

-Step  4:  Make  a  pattern  move. 

-Step  5:  Ascertains  that  the  first  goal  level  has  been  achieved. 

-Step  6:  Perfonn  the  gradient  move. 
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-Step  7:  Test  the  gradient  move  for  improvement. 

-Step  8:  If  no  improvement  found  in  step  7,  determine  whether  or  not  it  is  possible  to 
find  other  rays. 

-Step  9:  Reached  only  if  no  optimum  is  found  and  no  exploratory  move  or  ray  move  has 
found  a  direction  in  which  the  I  Vs  can  be  moved  to  improve  the  selected  goal  and  allows  the  user 
to  restart  the  search  at  a  new  initial  point  a  specified  distance  away  from  the  original  start  point. 

-Step  10:  Reached  only  if  no  optimum  is  found  and  no  exploratory  move  or  ray  move 
has  found  a  direction  in  which  the  IVs  can  be  moved  to  improve  the  selected  goal  then  the 
maximum  value  for  the  goal  has  been  determined  and  the  desired  goal  level  cannot  be  met. 
Decision  logic  either  tenninates  the  search  or  returns  to  step  two  with  a  new  desired  goal  level. 

The  advantages  and  disadvantages  of  this  algorithm  are: 

>  Advantages: 

-The  technique  is  efficient  because  of  its  systematic  search  procedures  and  because  it 
exploits  a  direction  of  change  in  the  IVs  until  it  no  longer  leads  to  an  improvement  in  the  goal 
levels. 

-The  technique  is  very  efficient  allowing  an  optimum  solution  to  be  reached  in  only  a 
fraction  of  the  computer  time  required  to  reach  a  non-linear  solution  when  solved  by 
enumeration. 

-Like  all  direct  search  optimization  methods,  this  technique  does  not  necessarily  require 
knowledge  of  the  functional  forms  of  the  goal  equations  although  it  does  require  that  goal  values 
be  obtained  that  are  dependent  upon  given  vectors  of  IVs. 

-The  method  does  not  require  linear  or  polynomial  approximation  of  the  fundamental 
forms  of  the  goal  equations  if  they  are  unknown,  as  would  be  the  case  of  multiple  response 
surface  methods  are  being  used.  Thus  the  method  can  be  tied  directly  to  the  output  of  a  non- 
Monte-Carlo  simulation  model,  and  the  solution  found  by  this  technique  will  be  a  solution  to  the 
simulation  model,  not  a  solution  to  an  approximation  of  it. 

>  Disadvantages: 

-The  algorithm  does  not  guarantee  that  the  global  optimal  solution  will  be  found. 
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-There  is  no  guarantee  that  the  method  is  the  most  efficient  in  the  sense  of  requiring  the 
fewest  number  of  point  evaluations  of  simulations  in  order  to  achieve  an  optimum. 

-The  technique  ignores  the  stochastic  behavior  of  the  simulation  response  on  the 
performance  of  the  algorithm. 

-The  model  algorithm  has  a  restriction  of  dealing  only  with  integer  problems  even 
though  there  are  large  numbers  of  realistic  problems,  which  are  non-integer. 

2.4. 4. 3.  Response  Surface  Methodology. 

Response  Surface  Methodology  (RSM),  originally  developed  by  Box  and  Wilson  in 
1951,  has  emerged  as  one  of  the  primary  tools  for  determining  optimum  values  of  the 
experimental  space  and  is  the  topic  of  a  wide  variety  of  discussion  in  the  literature.  RSM  was 
specifically  developed  to  provide  data  for  satisfying  the  needs  to  collect  data  in  a  more  efficient 
manner  than  conventional  factorial  experiments.  Its  purpose  is  to  collect  a  “foundation”  of 
information  that  can  be  useful  for  predicting  perfonnance  when  system  equipment  parameters 
are  not  currently  known  and  to  detennine  what  combination  of  these  parameters  can  optimize 
overall  system  performance  where  system  performance  must  be  expressed  as  a  function  of 
equipment  parameters,  where  the  independent  factors  are  measurable  on  a  quantitative, 
continuous  scale,  and  interaction  and  higher  order  terms  may  be  included  (Simon,  1968).  As 
opposed  to  traditional  ANOVA  factorial  and  fractional  factorial  designs,  RSM  focuses  on 
determining  the  functional  relationship  that  exists  between  the  response  and  specified 
continuous,  quantitative  factors,  rather  than  merely  detennining  the  significance  of  the  various 
individual  factors  (Clark  and  Williges,  1972b;  Clark  and  Williges,  1973). 

As  a  tool  for  gradient  estimation  and  sensitivity  analysis  in  discrete  simulation,  RSM 
possesses  noteworthy  advantages  in  comparison  to  other  techniques.  RSM  is  based  on  well- 
known  principles  of  regression  analysis  and  analysis  of  variance  and  is  more  completely 
developed  than  recent  extensions  of  techniques  such  as  perturbation  analysis  and  likelihood  ratio 
methods.  The  main  disadvantage  of  RSM  is  the  cost  of  making  all  of  the  simulation  runs 
required  by  the  experimental  design.  This  cost  can  be  prohibitive  when  the  response  of  interest 
has  a  large  variance  so  that  excessive  run  lengths  are  required.  This  naturally  invokes  the 
interest  for  searches  for  effective  variance  reduction  techniques  that  can  be  incorporated  into  the 
overall  experimental  design.  One  distinct  advantage  of  RSM  is  that  the  Monte  Carlo  method  of 
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control  variates  can  be  effectively  used  in  simulation  based  RSM  studies  with  a  negligible 
computational  overhead  (Wilson,  1987). 

The  RSM  methodology  involves  the  classical  application  of  experimental  design  based 
on  standard  least  squares  theory.  Generally  in  simulation-based  designs,  it  is  assumed  that  the 
response  surface  resulting  from  the  continuous  parameters  of  the  simulation  model  is  smooth 
enough  to  be  described  by  either  a  first  or  second-degree  polynomial  over  the  region  of  interest. 
The  methodology  is  sequential  in  nature  with  each  successive  experiment  building  on  the  results 
and  insights  of  earlier  experiments.  It  is,  therefore,  ideally  suited  to  simulation  because  of  the 
relative  ease  with  which  data  can  be  obtained  in  the  simulation  context  (Hood  and  Welch,  1993). 

Often  it  is  desired  to  utilize  the  simulation  model  to  attempt  to  find  the  optimum 
conditions  for  operating  the  system.  RSM  represents  a  body  of  techniques  by  which  the 
optimum  set  of  system  conditions  is  determined.  A  procedure  for  employing  a  first  order 
response  surface  approach  is  (Biles  and  Ozmen,  1987): 

-1 .  Identify  the  known  experimental  region. 

-2.  Perfonn  simulation  trials  at  each  of  the  experimental  design  points  and  record  the 
responses. 

-3.  Apply  the  appropriate  mathematical  programming  technique  to  locate  the  next  center 
point  in  the  search. 

-4.  Repeat  steps  1-3  until  an  “optimum”  solution  is  located.  It  may  be  necessary  to  add 
design  points  to  complete  a  second  order  response  surface  design  to  test  the  optimum  solution. 

Some  assumptions  and  characteristics  of  RSM  are  (Kleijnen,  1998): 

•  RSM  assumes  that  the  decision  variables  are  quantitative. 

•  RSM  assumes  a  single  type  of  response. 

•  RSM  relies  on  first  and  second  order  polynomial  regression  meta  models  or 
response  surfaces.  The  responses  are  assumed  to  have  white  noise. 

•  RSM  uses  classical  designs. 

•  RSM  adds  to  regression  models  and  DOE  the  mathematical  (not  statistical) 
technique  of  steepest  ascent;  that  is,  the  estimated  gradient  determines  in  which 
direction  the  decision  variables  are  changed. 
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•  RSM  uses  the  mathematical  technique  of  canonical  analysis  to  analyze  the  shape 
of  the  optimal  region:  Does  that  region  have  a  unique  maximum,  a  saddle  point, 
or  a  ridge  (stationary  points)? 

In  a  parallel  processing  system  RSM  approach,  the  desire  is  to  determine  the  2n'p  +  2N  + 

P  design  points  in  a  central  composite  design  and  to  systematically  allocate  the  determination  of 
these  points  to  P  client  processors  (Biles  and  Kleijnen,  1999). 

A  generalized  outline  for  applying  the  RSM  methodology  to  optimization  problems  is 
(Hood  and  Welch,  1993): 

•  Select  an  initial  experimental  region. 

•  Fit  a  first  order  model  to  the  region. 

•  Check  to  see  if  the  first  order  model  is  a  reasonable  fit  and  that  a  minimum  does 
not  fall  within  the  region. 

•  If  the  first  order  model  is  valid,  follow  the  path  of  steepest  descent  to  an  estimated 
minimum  of  the  function  along  the  path. 

•  Repeat  this  procedure  for  a  region  about  this  minimum  point. 

•  If  it  is  determined  that  it  is  likely  that  a  minimum  exists  within  the  region  then, 
augment  the  design  and  fit  a  second  order  model. 

•  Analyze  the  model  to  determine  whether  or  not  a  minimum  exists  and  continue. 

The  first  order  response  surface  methods  attempt  to  accomplish  experimentally  what  the 

“method  of  steepest  ascent”  accomplishes  computationally.  From  a  selected  point  in  the 
experimental  space  a  designed  experiment  is  conducted  with  a  simulation  trial  at  each  design 
point  to  estimate  the  gradient  direction.  Simulation  trials  are  then  conducted  at  points  along  this 
direction  to  a  new  point,  which  represents  the  best  solution  obtained  along  the  gradient  direction. 
The  gradient  direction  is  then  estimated  by  placing  an  appropriate  first  order  experimental 
design,  such  as  a  2n  factorial,  2n'p  fractional  factorial,  or  n  -  dimensional  simplex  design  around 
the  current  point  of  analysis.  A  simulation  trial  is  perfonned  at  each  point  in  the  selected 
experimental  design.  From  these  observations  a  multiple  linear  regression  model  can  be 
estimated.  A  simulation  trial  is  conducted  as  each  design  point  in  the  selected  first  order  design 
and  observations  are  recorded  at  each  design  point.  Multiple  linear  regression  is  applied  to  each 
observation  (assuming  independence  among  the  responses)  producing  a  model  for  each  response 
(Biles  and  Swain,  1979). 
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These  estimates  are  then  employed  in  an  optimization  scheme  to  produce  an  improved 
solution.  Simulation  optimization  procedures  that  can  be  employed  for  this  purpose  include 
Box’s  Complex  Search,  Rosen’s  gradient  projection  method  to  evaluate  a  first  order  response 
surface  procedure,  a  central  composite  design  coupled  with  a  computational  version  of  Box’s 
complex  search  for  a  second  order  response  surface.  One  point  that  this  author  notes,  which  may 
seem  minor  but  is  the  type  of  thing  that  often  consumes  an  inordinate  amount  of  time  and  energy, 
is  that  computer  programs  for  the  optimization  procedures  and  the  computer  programs  from  the 
simulation  must  often  be  “custom  fitted”  together  in  order  to  perfonn  this  analysis  (Biles  and 
Swain,  1979). 

Another  statement  of  the  generalized  procedure  to  be  followed  for  the  case  of  employing 
a  first  order  response  surface  approach  to  the  multiple-response  simulation  problem  is  (Biles  and 
Swain,  1979): 


1 .  Identify  the  known  experimental  region.  Select  a  starting  point  within  the 
region.  With  the  initial  point  as  its  center,  array  an  orthogonal  first  order 
response  surface  design  within  a  selected  design  radius. 

2.  Perform  simulation  trials  at  each  of  the  N  experimental  design  points  and 
record  the  responses.  Using  multiple  linear  regression,  fit  linear  models  to 
the  responses. 

3.  Apply  the  appropriate  mathematical  programming  technique  to  locate  the 
next  center  point  in  the  search. 

4.  Repeat  steps  1-3  until  an  “optimum”  solution  is  located. 

A  summary  of  advantages  of  the  use  of  RSM  for  computer  simulation  experimentation  is 
(Hood  and  Welch,  1993): 

o  It  is  sequential  so  it  matches  well  the  ready  availability  of  data  in  the 
simulation  context. 

o  It  is  sequential  both  in  the  model  fitting  and  estimation  at  each 

experimental  stage  and  in  the  generation  of  a  sequence  of  experimental 
stages. 

o  It  produces  confidence  intervals  on  the  estimates  of  interest  and  powerful 
diagnostics  for  the  model  fitting. 

o  It  provides  a  solid  theoretical  and  intuitive  base  in  classical  regression 
theory  for  the  experimental  process. 
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2.4. 4.4.  Box’s  Complex  Method  of  Constrained  Optimization. 

This  is  a  method  for  finding  the  maximum  of  a  general  non-linear  function  of  several 
variables  within  a  constrained  region  that  is  described  to  be  efficient  when  compared  with 
existing  methods  when  the  required  optimum  lies  on  one  or  more  constraints.  The  definition  of  a 
constrained  optimum  is  meant  one  for  which  the  solution  corresponds  to  certain  variables  lying 
at  the  edges  of  their  permissible  ranges.  If  this  is  not  the  case  a  method  with  no  provision  for 
bounding  the  variables  will  produce  the  same  result.  The  constrained  complex  method  searches 
for  the  maximum  value  of  a  function  subject  to  M  constraints  where  the  lower  and  upper 
constraints  are  either  constants  or  functions. 

This  approach,  using  gradient  based  optimization,  utilizes  successive,  alternative  phases, 
with  the  first  phase  consisting  of  a  first  order  experimental  design  which  estimates  an  “optimal” 
improving  direction  and  a  second  phase  determining  the  “optimal”  step  along  this  direction. 

(Box,  1965) 

+  Advantage:  Each  phase  lends  itself  nicely  to  the  assignment  of  a  set  of  S  simulation 
trials  to  P  parallel  processors. 

+  Disadvantages: 

o  Requires  a  parallel  computer  to  execute. 

o  The  lack  of  the  ability  to  perform  efficient  unconstrained  optimization. 

o  Continuation  of  the  search  when  one  or  more  constraints  become 
effective. 

o  Some  of  the  randomly  generated  points  will  be  relatively  remote  from  the 
initial  point  and  may  be  in  the  vicinity  of  a  higher  peak. 

2.4.4. 5.  Gradient  Based  Optimization  Approach. 

This  is  an  optimization  approach  that  takes  advantages  of  the  simultaneous  execution  of 
P  simulations  trials  on  parallel  processors.  Gradient  -  based  optimization  utilizes  successive, 
alternating  phases.  The  first  phase  consists  of  a  first  order  experimental  design  that  estimates  an 
“optimal”  improving  direction  and  a  second  phase  that  determines  the  “optimal”  step  in  the 
direction  of  the  first  phase.  An  advantage  of  this  technique  is  that  each  phase  can  be  tailored  to 
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the  assignment  of  a  set  of  S  simulation  trials  to  P  parallel  processors  (Biles  and  Kleijnen,  1999). 
However,  experimental  studies  comparing  actual  human  perfonnance  to  a  gradient  algorithm 
based  on  a  two  variable  optimization  task  found  the  algorithm  to  always  be  inferior  to  the  human 
operator  (Laughery  and  Drury,  1979). 

2.4.4. 6.  Genetic  Algorithms. 

The  Genetic  Algorithms  (GA)  approach  is  where  each  client  processor  is  assigned  its 
own  initial  random  number  seed  and  a  GA  search  is  undertaken  completely  independently  on 
each  processor.  GA  concepts,  developed  by  John  Holland,  are  based  on  natural  evolution 
phenomena  where  only  the  strongest  individuals,  i.e.,  the  best  adapted  to  the  environment, 
survive.  GA  algorithms  begin  their  search  of  the  optimal  solution  from  a  set  of  potential 
solutions  (i.e.,  individuals  represented  by  their  chromosomes),  which  is  called  the  population. 

An  initial  population  is  made  that  evolves  towards  a  population,  which  contains  the  best  solution. 
From  generation  to  generation,  the  population  contains  better  and  better  solutions  of  the  problem 
and  converges  to  an  optimum  (Pierreval  et  al.,  1995). 

The  GA  approach  greatly  reduces  the  interaction  between  the  Simulation  Manager  and 
the  clients.  Under  this  approach  each  client  processor  is  assigned  its  own  initial  random  number 
seed  and  an  independent  GA  search  is  undertaken  on  each  processor.  Each  of  the  P  processors 
acts,  analogically,  as  an  island  continent  for  the  purposes  of  evolution,  where  the  evolutionary 
process  takes  place  as  if  it  were  completely  unaware  of  the  existence  of  its  “neighbor” 
continents.  After  a  complete  GA  search  has  taken  place  on  each  of  the  P  client  processors,  the 
Simulation  Manager  receives  the  collective  statistical  results  and  determines  the  best  solution 
(Biles  and  Kleijnen,  1999). 

In  a  formal  sense,  data  analysis  is  not  the  main  field  of  application  of  GA.  They  should  be 
viewed  rather  as  a  powerful  technique  for  solution  of  various  combinatorial  or  optimization 
problems.  Nevertheless,  GA  are  applicable  for  datamining.  The  name  of  the  method  derives 
from  its  similarity  to  the  process  of  natural  selection.  Let  the  problem  be  to  find  a  solution  to  a 
question  that  would  be  the  most  optimal  from  the  point  of  view  of  a  certain  criterion.  Assume 
that  each  solution  can  be  exhaustively  described  by  some  set  of  numerical  or  non-numerical 
parameters.  For  example,  if  the  task  is  to  select  a  fixed  number  of  perfonnance  parameters 
influencing  the  C2  perfonnance  the  most,  then  the  names  of  these  parameters  comprise  such  a 
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descriptive  set.  One  can  think  of  this  set  as  of  a  set  of  chromosomes  detennining  qualities  of  an 
“organism”-a  solution  of  the  problem.  Following  this  analogy,  values  of  parameters  determining 
a  solution  correspond  to  genes.  A  search  for  the  optimal  solution  is  similar  then  to  the  process  of 
evolution  of  a  population  of  organisms,  where  each  organism  is  represented  by  a  set  of  its 
chromosomes  (Megaputer,  2002).  The  four  main  components  of  GA  are  chromosome 
representation,  evaluation  function,  genetic  operators  and  initialization  of  the  population.  When 
compared  to  other  rule  discovery  methods  such  as  artificial  neural  networks  and  statistical 
models,  the  advantage  of  techniques  such  as  genetic  algorithms  is  that  the  rules  evolved  are  self- 
explanatory  (Lopes,  Pacheco,  Vellasco,  and  Passos,  1999). 

2.4.4. 7.  Parallel  Evolutionary  Algorithm  Approach. 

Evolutionary  algorithms  (EA)  is  an  extension  of  GA  that  concern  the  coding  of 
chromosomes,  the  genetic  operators  or  the  selection  techniques.  Parallel  Evolutionary 
Algorithms  (PEA)  is  EA  implemented  on  a  parallel  processing  computer.  In  PEA  the  selection  is 
replaced  by  a  local  search,  generally  considered  as  a  “local  hill-climbing,”  where  each 
chromosome  tries  to  improve  its  fitness  by  itself  in  a  neighborhood.  An  example  of  a  fine 
grained  PEA  sequence  is  (Pierreval  et  ah,  1995): 


Step  0:  Define  a  genetic  representation  of  the  problem. 

Step  1 :  Create  an  initial  population. 

Step  2:  Each  chromosome  does  local  hill  climbing. 

Step  3:  Each  chromosome  selects  a  neighbor  for  mating. 

Step  4:  An  offspring  is  created  with  genetic  operators  working  on  the  genotypes 
of  its  parents. 

Step  5:  The  offspring  does  local  hill  climbing.  It  replaces  the  parent,  if  it  is  better 
than  some  criterion. 

Step  6:  If  not  finished,  return  to  Step  3. 


2.4.5.  Sequential  Experimentation. 

Experimental  designs  based  on  full  factorial  evaluations  very  quickly  become  inefficient 
to  the  point  of  being  impractical  to  conduct  as  the  number  of  factors  goes  up.  Considering 
experiments  with  only  two  treatment  levels,  for  example,  five  factors  would  generate  25  =  32 
treatment  combinations.  However,  many  experiments  in  human  factors  can  involve  10,  or  20  or 
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more  IV  factors.  A  ten  factor  -  two  level  full  factorial  experiment  involves  2 10  =  1024 
treatments  to  be  conducted.  While  certain  techniques  such  as  fractional  factorial  replicates  can 
reduce  these  numbers,  the  situations  that  occur  when  different  factors  have  different  numbers  of 
treatment  levels  greater  than  two  and  when  all  the  treatments  cannot  be  conducted  at  the  same 
time,  place,  or  by  the  same  experimenter,  many  times  result  in  the  fact  that  no  explicit  data 
reduction  techniques  are  available  that  can  gather  the  required  data.  When  there  are  large 
numbers  of  factors  to  be  evaluated  over  extended  periods  of  time  among  different  locations  and 
by  multiple  experimenters  then  the  technique  of  sequential  experimentation  becomes  viable. 

In  the  mid  1970s  investigations  were  undertaken  to  examine  traditional  experimental 
design  methods  as  they  applied  to  human  factors  engineering  (then  referred  to  as  engineering 
psychology)  where  the  two  distinct  approaches  of  experimental  and  correlational  empirical 
investigation  were  compared.  Arguments  were  made  for  merging  the  more  productive  features 
of  each  into  a  new  research  paradigm  for  experimental  psychology  called  the  sequential  process. 
This  proposed  systematic  multi-factor  experiments  that  could  be  perfonned  economically.  With 
this  paradigm  the  total  data  collection  process  for  deriving  equations  for  all  critical  variables 
affecting  the  system  under  investigation  would  be  less  than  that  used  for  a  4  or  5  factor 
experiment  using  traditional  means.  This  paradigm  was  described  in  five  phases  (Simon, 

1977b): 

•  Define  the  systems  research  problem. 

•  Identify  the  critical  variables. 

•  Develop  a  response  surface. 

•  Refine  the  regression  equation. 

•  Verify  the  experimental  results. 

Further  proposals  were  made  that  described  methods  for  constructing  Resolution  IV 
screening  designs  ascribed  as  being  robust  to  linear,  quadratic,  and  cubic  trend  effects.  Complete 
designs  of  up  to  32  variables  were  presented  using  this  technique.  These  screening  designs,  in 
some  ways  a  precursor  and  in  some  ways  a  component  of  sequential  experimentation,  was 
described  as  a  class  of  fractional  factorials  with  systematic  data  collection  plans  that  enabled  the 
effects  of  large  numbers  of  factors  to  be  estimated  economically.  The  purpose  of  screening 
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designs  was  to  identify  the  most  important  factors  not  to  obtain  an  accurate  representation  of  the 
full  experimental  space.  However,  the  full  benefit  of  screening  designs  was  that,  once  the  most 
important  factors  were  identified  the  same  data  can  be  further  utilized  by  supplementing  the 
original  experiments  by  a  relatively  few  additional  observations  with  new  experimental 
conditions  that  would  have  the  ability  to  describe  the  response  surface  that  would  be  capable  of 
representing  the  full  experimental  space  of  the  original  large  number  of  IVs  (Simon,  1977a). 

As  computer  simulations  began  providing  an  ever  more  capable  tool  for  complex  human 
performance  investigation,  it  was  noted  that  the  primary  difficulty  of  using  simulation  for 
experimental  design  could  be  summarized  as  too  many  factors  and  too  few  runs.  Factor 
screening  methods  were  acknowledged  as  being  able  to  reduce  the  number  of  factors  in  large 
experiments  where  the  basic  aim  was  to  efficiently  and  effectively  classify  as  active  or  as 
inactive  the  K  factors  under  investigation.  Two  approaches  for  this  were  described  as  the  use  of 
expert  judgment  where  the  analyst  decides  which  factors  are  important  and  which  are  not,  and 
group  screening  where  “group  factors”  are  created  by  partitioning  the  individual  factors  into  a 
number  of  groups  (Smith  and  Mauro,  1981). 

Building  on  the  work  of  Simon  and  others,  efforts  were  undertaken  to  describe  systematic 
methods  that  could  be  applied  to  human  performance  system  studies  that  could  only  be 
realistically  described  by  large  numbers  of  factors  that  could  number  100  or  more  (Han  et  ah, 
1997;  Williges,  1981;  Williges  and  Williges,  1989;  Williges  et  ah,  1993).  The  goal  of  sequential 
experimentation  is  to  develop  integrated  empirical  models  describing  user  performance  as  a 
function  of  a  large  number  of  IVs  investigated  across  a  series  of  small  studies  (Williges  et  ah, 
1993).  Three  general  approaches  for  conducting  empirical  research  with  systems  that  deal  with 
complex  relationships  among  many  IVs  include  the  use  of  direct  observations  of  the  operational 
system  or  prototype  if  possible,  the  use  of  analytic  methods  of  modeling  and  computer 
simulation,  and  real  time  (human  -  in  -  the  -  loop)  simulation  with  manipulation  of  various 
system  parameters  according  to  experimental  designs  (Williges,  1981).  Some  of  the 
experimental  design  methods  that  can  be  used  to  provide  for  efficient  data  collection  and 
reduction  in  complex  system  design  and  experimentation  include: 

•  Single  observation  factorials  where  only  one  observation  is  made  in  each 
treatment  combination. 
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•  Hierarchical  designs  where  factors  other  than  subjects  are  nested  causing  the  level 
of  one  factor  to  appear  at  only  one  level  of  another  factor  which  results  in  a 
hierarchical,  or  pyramidal,  shape  of  the  treatment  conditions. 

•  Blocking  designs  where  the  data  is  collected  in  stages  or  blocks  over  time  or 
between  locations. 

•  Fractional  factorial  designs  where  the  nature  and  extent  of  confounding  is 
determined  by  specifying  the  subset  of  treatment  combinations  to  be  observed 
from  the  full  factorial  design. 

•  Empirical  model  building  where  the  researcher’s  primary  aim  is  the  determination 
of  a  quantitative  relationship  between  human  performance  and  quantitative 
system  parameters. 

•  Central-Composite  Designs  where  the  aim  is  to  determine  the  optimal 
combination  of  various  factors  from  the  total  system  design. 

Some  alternative  sequential  design  procedures  include  the  use  of  random  and  independent 

selections  of  experimental  data  points,  a  series  of  single  factor  experiments,  and  /  or  the  use  of 

the  method  of  steepest  ascent  in  response  surface  exploration  (Williges,  1981). 

The  stages  involved  in  the  use  of  sequential  research  are  shown  in  Figure  17  and  include 
selecting,  describing,  and  optimizing  I  Vs.  The  goal  of  this  research  strategy  is  to  obtain 
estimates  of  the  functional  relationships  of  a  large  number  of  IVs  in  a  realistic,  efficient,  and 
systematic  manner. 


Considerations  Considerations  Considerations 

Initial  Selection  Small  Experiments  Research  Methods 

Screening  Studies  Empirical  Models  Data  Analysis 

Data  Bridging 

Figure  17  -  Stages  In  Sequential  Research  Paradigm 

(Williges,  1981;  Williges,  2001) 

(Used  With  Permission  of  the  Author) 

Figure  17  shows  the  stages  involved  in  the  preparation  of  IVs  for  sequential  research. 
The  methods  involved  for  each  stage  include  (Williges,  2001): 

•  Stage  1 :  Brainstorming,  prototyping,  subjective  ratings,  literature  review,  and 
screening  studies. 
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•  Stage  2:  2k  factorials,  x/2  replicates,  central  composite  design,  polynomial 

regression,  integrated  database. 

•  Stage  3:  Steepest  ascent,  random  selection,  partial  derivatives. 

The  first  stage  of  this  methodology  involves  the  identification  of  all  possible  variables 
that  could  affect  human  performance  in  the  system  being  evaluated.  The  second  stage  is  the 
conduct  of  several  small  experiments  to  gather  the  necessary  data  to  evaluate  the  variables 
identified  in  stage  1  (Diamond,  1981).  The  experimental  designs  that  are  used  in  this  stage 
generally  only  focus  on  main  effects  and  two-way  interactions.  This  is  because,  in  human 
factors  research,  it  generally  is  not  feasible  to  interpret  human  performance  interactions  above 
second  order.  While  interactions  of  variables  such  as  “talking  on  a  cell  phone”  and  “driving  an 
automobile”  can  generally  be  evaluated,  the  interpretation  of  a  third  variable  such  as  “reading  an 
information  display”  generally  is  not  possible  in  human  factors  research.  For  this  reason 
Resolution  V  experiments  that  examine  the  main  effects  of  the  individual  variables  and  two-way 
interactions  are  the  highest  level  experiments  that  are  generally  employed.  The  key  to  these 
experiments  is  efficient  experimental  designs  that  allow  for  data  collection  economy.  The  third 
stage  of  sequential  design  is  to  determine  the  optimum  combination  of  levels  of  the  important 
IVs.  From  this  an  empirical  model  is  developed  that  integrates  the  relationships  from  the  most 
important  IVs.  Optimization  procedures  can  include  response  surface  methodology,  ridge 
analysis,  and  mixed  integer  programming  (Williges,  1981;  Williges  et  al.,  1993). 

Before  beginning  a  program  of  sequential  research  a  plan  must  be  developed  that 
considers  the  constraints  and  region  of  interest  for  the  project.  The  selection  of  IVs  for  a 
sequential  research  plan  involves  both  the  initial  design  factors  and  screening  studies  to  reduce 
the  number  of  factors  to  a  reasonable  size  for  the  experimentation  and  model  building.  Once  the 
set  of  IV s  to  be  used  in  the  experiment  has  been  identified  from  the  larger  set,  a  series  of  small 
experiments  is  conducted  to  gather  data  to  describe  the  effects  of  these  variables.  The  results 
from  each  of  these  small  experiments  is  then  combined  together  by  a  data  bridge  so  that  the 
functional  relationships  pertaining  to  all  the  IVs  can  be  identified  and  analyzed.  From  this  single 
data  set  an  integrated  empirical  model  can  be  generated.  Finally,  the  last  stage  of  the  sequential 
design  procedure  is  to  determine  the  optimum  combination  of  important  design  variables  (Han  et 
al.,  1997;  Williges  et  al.,  1993). 
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When  embarking  on  studies  involving  sequential  experimentation,  distinct  consideration 
must  be  given  on  what  types  of  experimental  designs  might  be  appropriate  for  the  individual  data 
collection  runs.  First,  discussions  of  two  types  of  IVs  that  are  associated  with  sequential 
experimentation  are  presented  (Williges  et  ah,  1993): 

•  Dichotomous  variables  are  discontinuous  factors  encountered  in  human  factors 
research.  Examples  are  gender  (male  or  female)  or  availability  of  a  specific 
feature.  This  class  of  variables  is  evaluated  with  two  level,  2k,  factorial  designs. 
When  the  amount  of  data  precludes  its  full  collection  because  of  such  things  as 
time  or  budget  restraints  and  the  researcher  can  be  satisfied  with  a  Resolution  V 
experiment  where  only  the  main  effects  and  two  way  interactions  are  of  interest, 
then  fractional  factorial,  2k'p,  designs  can  provide  an  efficient  alternative  for 
simultaneous  consideration  of  multiple  factors. 

•  Continuous  variables  can  be  evaluated  using  central  composite  designs,  which 
provide  an  efficient  means  for  handling  this  class  of  variable.  Second  order 
models,  including  main  effects,  pure  quadratics,  and  two  way  interactions,  can  be 
estimated  with  the  CCD  design.  A  central  composite  design  consists  of  three 
portions:  1)  A  factorial  portion,  2k  or  2k'p,  2)  An  axial  portion,  and  3)  A  center 
point. 

If  the  researcher  can  assume  that  the  main  effects  and  two  way  interactions  are  of  the 
primary  importance  for  human  factors  research,  and  that  higher  order  effects  are  not  of  interest, 
then  useful  designs  for  sequential  experimentation  are  (Williges  and  Williges,  1989;  Williges  et 
al„  1993): 

1 .  Two-level  full-  or  fractional-factorial  designs  to  be  used  in  the  evaluation  of 
dichotomous  variables. 

2.  Central-composite  designs  to  be  used  to  evaluate  continuous  variables. 

3.  Data  bridging  designs  to  add  required  evaluations  for  the  investigation  of 
interactions  not  investigated  in  the  set  of  sequential  experiments.  These  data 
points  are  generated  with  the  view  to  minimize  multicollinearity  effects  that 
include  large  variances  of  coefficients,  poor  power  of  partial  -  F  tests,  incorrect 
signs  on  coefficients,  and  poor  prediction  performance.  Controls  on  serious 
multicollinearity  in  a  regression  model  are  implemented  by  insuring  that  the 
variance  inflation  factor  (VIF)  should  be  smaller  than  10  and  the  eigenvalue  ratios 
should  be  smaller  than  100  in  order  to  test  model  items  for  statistical  significance. 
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2.4.6.  Simulations  of  Human  System  Performance. 

Modeling  the  cognitive  processes  that  are  involved  in  infonnation  processing  is  central  to 
understanding  human  capabilities  and  limitations  in  computer  tasks  (Williges,  1987). 

Simulations  can  permit  a  much  more  precise  level  of  control  of  many  variables  than  could  ever 
be  realized  in  physiological  or  behavioral  experimental  paradigms  (Grossberg  and  Mingolla, 
1986).  The  significant  dimensions  of  the  human  system  include  the  task  dimension,  the 
performance-learning-development  dimension  and  the  individual  difference  dimension.  The 
human  infonnation  processing  system  is  a  serial  system  consisting  of  an  active  processor,  input 
(sensory)  and  output  (motor)  systems,  an  internal  long-tenn  and  short-term  memory  and  an 
external  memory.  There  is  general  agreement  that  human  memory  is  best  modeled  in  terms  of  a 
basic  division  between  active  short  tenn  memory  and  a  passive  storehouse  called  long  term 
memory  (Pew  and  Mavor,  1998e).  For  modeling  purposes  memory  can  be  broken  down  into 
three  broad  categories: 

•  Episodic,  generic,  and  implicit  memory. 

•  Short  tenn  and  working  memory. 

•  Long-term  memory  and  retrieval. 

Some  of  the  sources  of  information  providing  input  to  the  human  system  include  (Newell  and 
Simon,  1972): 

•  The  task  instructions  themselves. 

•  Previous  experience  with  the  same  task. 

•  Previous  experience  with  tasks  that  are  recognized  as  somehow  analogous  to  the 
given  task. 

•  Stored  programs  in  long-tenn  memory  of  substantial  generality  that  can  be 
applied  to  a  range  of  tasks. 

•  Stored  programs  in  long  term  memory  for  constructing  problem  spaces. 

•  The  course  of  problem  solving  itself. 

Human  behavior  is  a  complex  result  of  many  factors  that  include  environmental 
moderators  and  interpersonal  differences.  Human  behavior  moderators  include  (Pew  and  Mavor, 
1998a)  external  moderators  such  as  physiological  stressors  that  include  environmental  conditions 
such  as  temperature,  toxic  substances,  noise,  vibration,  etc,  and  other  conditions  such  as  physical 
work  and  fatigue-effects  of  continuous  performance.  They  may  also  include  cognitive  workload 
stressors,  which  are  described  as  a  consistent  human  adaptation  to  overwhelming  task  loads  in 
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information  processing  tasks  (such  as  command  and  control).  The  reaction  to  this  is  to  exclude 
some  portion  of  the  signal  flow  or  postpone  the  processing  response  until  the  peak  period  has 
passed.  Some  responses  to  cognitive  overload  are  dysfunctional.  In  this  case,  sequence  sense  is 
often  lost.  For  example,  where  a  tank  battalion  commander  receives  messages  regarding  unit 
losses  in  a  high  signal  flow  load,  the  initial  signals  may  become  overlooked  with  the  commander 
not  being  aware  of  the  true  extent  of  the  overall  losses. 

Internal  moderators  of  human  behavior  include  intelligence  such  as  ASVAB  scores  and 
level  and  type  of  expertise  and  cognitive  abilities.  If  a  task  is  within  the  physical  capability  of 
the  soldier,  the  cognitive  aspects  of  the  task  will  display  greater  variation  in  performance  than 
will  physical  aspects  that  are  frequently  practiced.  Personality  traits  frequently  judged  by 
observers  of  military  affairs  to  have  a  high  likelihood  of  salience  for  the  military  character 
include  need  achievement,  risk  taking  and  innovativeness  and  a  general  alertness  and  vigilance. 
Likewise,  emotions  such  as  anxiety,  obsessiveness,  and  depression  are  an  internal  moderator  as 
well  as  attitudes  and  expectancies  where  expectancies  are  a  product  of  the  interaction  between 
beliefs  and  attitudes.  Finally,  cultural  values  can  induce  a  profound  effect  on  the  behavior  of  the 
individual  warrior. 

Another  view  to  human  perfonnance  modeling  is  called  the  normative-descriptive 
approach  where  motivated  expert  human  decision  makers  strive  for  optimality  but  are 
constrained  from  achieving  it  by  their  inherent  perceptual  limitations  and  cognitive  biases.  This 
approach  includes  infonnation  processing  models  such  as  the  distributed  infonnation  processing 
(DIP)  model  that  predicts  how  team  members  weight  and  combine  sequential  infonnation  from 
distributed  sources.  The  hierarchical  information-processing  (HIP)  Model  predicts  how  a  leader 
in  a  hierarchical  team  combines  the  opinions  and  confidences  of  subordinates  to  solve  an  event 
identification  (or  hypothesis-testing)  problem.  It  also  includes  resource  allocation  models  such 
as  the  distributed  resource  allocation  and  management  model  (DREAM)  that  represents  the  time 
evolution  of  task  appearance  and  movement,  as  well  as  resource  flow,  within  a  dynamic 
programming  framework  and  then  obtains  a  sub  optimal  solution  to  the  formulated  optimization 
problem  and  the  team  distributed  scheduling  (TDS)  Model  that  examines  how  team  decision 
making  and  coordination  strategies  adapt  to  increased  task  load  and  resource  scarcity  under 
different  responsibility  structures  (Kleinman  et  ah,  1992). 
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There  are  three  major  categories  of  simulations  in  use  in  the  military.  These  are  training 
simulations  for  individual  combatants  or  leaders  and  teams.  The  second  type  is  analytical 
simulations  for  analyses  of  systems,  doctrine,  and  tactics  for  purposes  of  acquisition  and 
advanced  development.  Then  there  are  those  simulations  that  address  questions  associated  with 
improving  command  and  control  and  the  interoperability  of  joint  forces.  While  a  model  is 
defined  as  a  physical,  mathematical,  or  otherwise  logical  representation  of  a  system,  entity, 
phenomenon,  or  process,  a  simulation  is  defined  as  a  method  for  implementing  a  model  over 
time.  Types  of  simulations  in  use  in  the  military  include  (Pew  and  Mavor,  1998c): 

•  Live  simulations,  which  are  live,  field  exercises. 

•  Virtual  simulations  where  real  humans  operate  simulated  equipment  in  simulated 
environments. 

•  Constructive  simulations  where  simulated  people  operate  simulated  equipment 
and  may  or  may  not  be  in  real  time. 

Three  common  approaches  used  to  represent  the  C3  architecture  in  models  that  examine  the 
behavior  and  decision  making  of  organizational  units  are  the  rule  based  approach  the  network 
approach  and  the  Petri  Net  approach  (Pew  and  Mavor,  1998f). 

2.5.  Selected  Statistical  and  Experimental  Approaches. 

Investigations  of  the  order  of  magnitude  envisioned  for  the  COMPASS  paradigm,  where 
a  hundred  or  more  IV  may  be  evaluated  for  dozens  of  DV,  require  analytical  approaches  that 
transcend  normal  statistical  and  experimental  designs.  Multivariate  techniques  that  are 
augmented  by  datamining  approaches  are  seen  as  an  appropriate  venue  to  investigate  the  amount 
and  type  of  data  that  this  study  will  produce.  This  section  presents  a  review  of  some  of  these 
techniques. 

2.5.1.  Cluster  Analysis. 

Cluster  Analysis  (CA)  is  the  generic  name  for  mathematical  models  that  can  be  used  to 
find  out  which  objects  in  a  set  are  similar  (Romesburg,  1984).  It  also  allows  the  ability  to  find 
out  which  objects  in  a  set  are  dissimilar.  There  are  a  number  of  methods  of  CA  but  the  most 
widely  used  is  hierarchical  CA  (HCA).  Other  CA  methods  are  the  single  linkage,  the  complete 
linkage,  Ward’s  method,  and  the  centroid  clustering  method.  The  steps  in  HCA  are: 
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1 .  Collect  a  data  matrix  where  the  columns  consist  of  the  objects  to  be  cluster 
analyzed  and  the  rows  are  attributes  that  describe  the  objects. 

2.  Standardize  the  data  matrix  (optional). 

3.  Compute  the  values  of  a  resemblance  coefficient  as  a  measure  of  the  similarities 
among  all  the  pairs  of  the  data  objects  (from  the  standardized  data  matrix). 

4.  Process  the  values  of  the  resemblance  coefficient  using  a  clustering  method  which 
results  in  a  tree  diagram  or  dendrogram  that  graphically  shows  the  relationship 
between  the  similarities  among  all  the  pairs  of  objects. 

Clustering  can  be  generally  defined  as  a  problem  where  “N”  points  are  given  in  a 
dimensional  feature  space,  with  the  requirement  to  find  “d”  interesting  groups  of  points.  There  is 
no  definitive  way  to  quantify  what  “interesting”  means  in  this  context,  however,  many 
algorithms  assume  that  there  are  a  certain  number  of  expected  clusters,  “k,”  with  the  requirement 
to  find  these  clusters  so  as  to  minimize  some  error  metric.  Examples  of  applications  where 
clustering  is  applicable  include  classification  problems  in  machine  learning,  information  retrieval 
to  identify  concepts,  to  improve  the  presentation  of  web  search  results,  and  by  physicists  to  find 
the  spatial  grouping  of  stars  into  galaxies.  In  general  the  desire  is  to  find  relationships  in  the  data 
and  to  succinctly  model  the  data  distribution  (Palmer  and  Faloutsos,  1999). 

Applications  include  the  commercial  sector  where  data  from  billions  of  credit  card 
transactions  and  cash  register  receipts  are  processed  annually  in  order  to  analyze  purchasing 
patterns  and  uncover  evidence  of  fraud.  In  the  national  security  arena,  vast  amounts  of  electronic 
and  image  data  are  processed  in  near  real  time  with  the  requirement  to  search  for  features  of 
special  interest  (Joyce,  Abarbanel,  Callan,  Dally,  and  Dyson,  2000). 

Cluster  analysis  is  not  a  typical  statistical  test  but  is  a  collection  of  different  algorithms 
that  “put  objects  into  clusters.”  Statistical  significance  testing  is  not  appropriate  with  cluster 
analysis  even  in  cases  when  p-levels  are  reported  (as  in  k-means  clustering).  The  purpose  of 
cluster  analysis  is  to  join  together  objects  into  successively  larger  clusters,  using  some  measure 
of  similarity  or  distance.  Typically  this  clustering  is  shown  as  the  hierarchical  tree.  The  tree 
clustering  method  uses  the  dissimilarities  or  distances  between  objects  in  forming  the  clusters. 
Types  of  distances  that  can  be  used  with  cluster  analysis: 

o  Euclidean  distance-the  most  commonly  used  type  of  distance.  It  simply  is 
the  geometric  distance  in  the  multimensional  space. 
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o  Squared  Euclidean  distance-places  progressively  greater  weight  on  objects 
that  are  further  apart. 

o  City-block  (Manhattan)  distance-the  average  difference  across 
dimensions. 

o  Chebychev  distance-appropriate  when  the  desire  is  to  define  two  objects 
as  “different”  if  they  are  different  on  any  one  of  the  dimensions. 

o  Power  distance-where  the  desire  is  to  increase  or  decrease  the  progressive 
weight  that  is  placed  on  dimensions  on  which  the  respective  objects  are 
very  different. 

o  Percent  disagreement-used  for  data  for  the  dimensions  included  in  the 
analysis  are  categorical  in  nature. 

One  form  of  cluster  analysis  uses  k  means  clustering  where  the  expected  number  of 
clusters  is  predetermined  by  a  hypothesis  (StatSoft,  2002). 

2.5.2.  Datamining  and  Neural  Networks. 

The  merging  of  statistics,  machine  learning  and  database  management  has  resulted  in  the 
emerging  technology  area  called  datamining  (Clifton  and  Thuraisingham,  2001).  Datamining  is 
the  search  for  and  extraction  of  hidden  and  useful  patterns  of  information  and  structures  in  large 
multidimensional  datasets  that  were  originally  collected  for  another  purpose.  It  is  the  search  of 
observational  data  for  the  relationships  between  parameters  in  that  data  rather  than  the 
measurement  of  experimental  data.  A  major  limitation  of  datamining  is  that  the  search 
algorithms  chosen  for  the  search  may  miss  an  important  and  interesting  pattern  or  even  a  class  of 
patterns.  There  is  no  systematic  method  currently  known  to  preclude  this  (Ceruti  and  McCarthy, 
2000).  Datamining  techniques  are  used  to  extract  previously  unknown  information  from  a  large 
database  that  typically  can  house  millions  of  pieces  of  infonnation  about  customers  and  customer 
relationships.  It  is  the  process  of  posing  queries  and  extracting  infonnation  often  previously 
unknown  from  large  quantities  of  data.  Datamining  outcomes  include  forming  clusters  as  well  as 
making  associations  and  correlations  (Thruraisingham,  Clifton,  Maurer,  and  Ceruti,  2001).  Part 
of  the  problem  is  that  there  is  simply  not  enough  knowledge  about  the  many  factors  that  govern 
human  behavior  to  accurately  predict  results  of  such  actions  as  decision-making.  While 
datamining  is  no  panacea  to  overcome  this  deficiency,  it  can  provide  correlations  according  to 
behavior  patterns  and  can  help  guide  the  search  for  types  and  timing  of  decisions  that  are  made 
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but  cannot  accurately  predict  what  the  outcomes  of  the  decisions  will  be.  Behavioral  processes 
are  next  to  impossible  to  predict  with  a  high  degree  of  accuracy.  Emotions  make  it  possible  for  a 
person  to  make  a  decision  for  no  obvious  reason  and  then  to  change  his  mind  seconds  later.  This 
is  a  domain  where  a  mere  matter  of  seconds  can  make  the  difference  between  one  response  or 
another  (Wisner,  1999). 

Datamining  is  an  emerging  research  area  with  a  focus  to  extract  significant  patterns  or 
interesting  rules  from  large  databases  (Zaki,  Parthasarathy,  Li,  and  Ogihara,  1996).  The 
technique  relies  on  mathematical  algorithms  and  neural  networks  to  reach  into  relational  data 
bases  and  uncover  new  patterns  among  the  data  (Asbrand,  1997).  It  is  the  process  of  posing 
queries  and  extracting  information  often  previously  unknown  from  large  quantities  of  data  where 
the  outcomes  include  forming  clusters  as  well  as  making  associations  and  correlations 
(Thruraisingham  et  ah,  2001).  At  present  Evolutionary  Programming  is  the  youngest  and 
evidently  the  most  promising  branch  of  datamining.  The  underlying  idea  of  the  method  is  that  the 
system  automatically  formulates  hypotheses  about  the  dependence  of  the  target  variable  on  other 
variables  (Megaputer,  2002).  Datamining  is  the  process  of  extracting  patterns  as  well  as 
predicting  (previously  unknown)  trends  from  large  quantities  of  data  by  posing  (automatically) 
repeated  queries. 

One  classification  of  datamining  models  is  (Clifton  and  Thuraisingham,  2001): 

•  Tree  Classification. 

•  Polynomial  Regression. 

•  General  Regression. 

•  Association  rules. 

•  Neural  Networks. 

•  Clustering. 

Nonlinear  Regression  Methods  are  based  on  searching  for  a  dependence  of  the  target 
variable  on  other  variables  in  the  form  of  function  of  some  predetermined  fonn.  For  example,  in 
one  of  the  most  successful  implementation  of  algorithms  of  this  type,  group  attribute  accounting 
method,  a  dependence  is  sought  in  the  fonn  of  polynomials.  Such  methods  must  provide 
solutions  with  a  larger  statistical  significance  than  neural  networks  do.  An  obtained  formula,  a 
polynomial,  is  more  suitable  for  analysis  and  interpreting  in  principle  (in  reality  it  is  usually  still 
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too  complex  for  that).  Thus  this  method  is  useful  in  providing  reliable  solutions  in  such  involved 
applications  as  financial  markets  or  medical  diagnostics  (Megaputer,  2002). 

Of  these  datamining  approaches,  neural  networks  and  clustering  are  among  the  more 
widely  used.  Neural  networks  are  computer  programs  whose  architecture  is  patterned  after  the 
neurons  in  the  human  brain.  It’s  made  up  of  a  web  of  electronic  neurons  that  send  signals  to 
each  other  through  thousands  of  connections,  which  are  adjusted  up  or  down  as  the  program 
learns  a  particular  application  (Anonymous,  1997).  It  can  be  defined  as  a  distributed 
computational  system  composed  of  individual  processing  nodes  that  operate  in  parallel  and  are 
interconnected  according  to  a  specific  architecture.  Learning,  in  the  context  of  neural  networks, 
is  the  capacity  to  self-modify  connection  strengths  and  processing  element  parameters  (Reilly 
and  Cooper,  1995).  Neural  networks  have  a  long  and  interesting  history  dating  back  to  the  1950s 
(Darling,  1997).  Neural  networks  are  approximation  tools  that  learn  the  relationships  between 
IVs  and  DVs,  much  like  regression  or  other  more  traditional  approaches.  The  principal 
difference  between  neural  networks  and  statistical  approaches  is  that  neural  networks  make  no 
assumptions  about  the  statistical  distribution  of  properties  of  the  data  and  therefore  tend  to  be 
more  useful  in  practical  situations  (Smith  and  Gupta,  2000). 

Types  of  Neural  Network  Models  include: 

•  Multilayered  feedforward  neural  networks  (MFNN)-appropriate  for  solving 
problems  that  involve  learning  the  relationships  between  a  set  of  inputs  and 
known  outputs.  They  are  a  supervised  learning  technique  in  the  sense  that  they 
require  a  set  of  training  data  in  order  to  learn  the  relationships.  The  neurons  are 
connected  in  layers,  and  the  weights  are  modified  throughout  the  algorithm  to 
reflect  the  leaning  process. 

•  Hopfield  neural  networks  (HNN)-a  fully  interconnected  system  of  N  neurons 
where  the  weights  of  the  network  are  fixed  and  symmetric  and  store  information 
about  the  memories  or  stable  states  of  the  network.  HNNs  are  principally  used  to 
solve  optimization  problems. 

•  Self-organizing  neural  networks  (SOFM)-similar  to  MFNNs  but  are  used  as  a 
clustering  technique  when  no  training  data  are  available. 

Neural  networks  (MFNNs  and  SOFMs)  are  used  by  most  commercial  datamining  packages  such 
as  the  SAS  Enterprise  Miner  and  the  IBM  Intelligent  Miner. 
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The  functioning  of  a  biological  brain  depends  on  networks  of  nerve  cells,  called  neurons 
that  are  connected  with  each  other  by  links  called  synapses.  Some  of  the  functions  of  a 
biological  brain  can  now  be  mimicked  with  computer-based  models  of  neural  networks. 

However,  models  of  neural  networks  that  are  simulated  in  computers  are  far  simpler  than  the 
highly  complex  and  often  messy  neural  systems  that  have  been  devised  by  nature  (Sejnowski, 
1998).  Artificial  Neural  Networks  are  relatively  crude  electronic  models  based  on  the  neural 
structure  of  the  brain  that  follows  the  realization  from  current  research  that  shows  that  brains 
store  information  as  patterns.  The  term  “crude”  is  utilized  with  the  knowledge  that  biological 
human  brains  have  about  100  billion  neurons.  Each  neuron  can  connect  with  up  to  200,000  other 
neurons  with  a  typical  connection  pattern  of  around  1,000  to  2,000.  Biological  neurons  receive 
inputs  from  other  sources,  combines  them  in  some  way,  performs  a  generally  nonlinear  operation 
on  the  result,  and  then  outputs  the  final  result  (Anderson  and  McNeill,  1992).  Neural  Networks 
is  a  large  class  of  diverse  systems  whose  architecture  to  some  extent  imitates  structure  of  live 
neural  tissue  built  from  separate  neurons.  One  of  the  most  widespread  architectures,  multilayered 
perceptron  with  back  propagation  of  errors,  emulates  the  work  of  neurons  incorporated  in  a 
hierarchical  network,  where  the  input  of  each  neuron  of  the  next  layer  (narrower)  is  connected 
with  the  outputs  of  all  neurons  of  the  previous  (wider)  layer.  Analyzed  data  are  treated  as  neuron 
excitation  parameters  and  are  fed  to  inputs  of  the  first  layer.  These  excitations  of  a  lower  layer 
neurons  are  propagated  to  the  next  layer  neurons,  being  amplified  or  weakened  according  to 
weights  (numerical  coefficients)  ascribed  to  corresponding  intraneural  connections.  As  a  final 
result  of  this  process,  the  single  neuron,  comprising  the  topmost  neuron  layer,  acquires  some 
value  (excitation  strength)  which  is  considered  to  be  a  prediction-the  reaction  of  the  whole 
network  to  the  processed  data  (Megaputer,  2002). 

Artificial  neurons  are  called  “processing  elements”  and  neural  networks  are  the  simple 
clustering  of  the  primitive  artificial  neurons.  There  are  two  primary  ways  that  neural  networks 
are  employed.  These  are  called  supervised  and  unsupervised  “training”  (Sejnowski,  1998).  In 
supervised  training  both  the  inputs  and  outputs  are  provided  and  the  network  processes  the  inputs 
and  compares  its  resulting  outputs  against  the  desired  outputs.  Errors  are  then  propagated  back 
through  the  system,  causing  the  system  to  adjust  the  weights,  which  control  the  network.  The 
process  is  repeated  over  and  over  until  a  threshold  is  reached.  Unsupervised  or  adaptive  training 
is  where  the  network  is  provided  with  inputs  but  not  with  desired  outputs.  The  system  itself  must 
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then  decide  what  features  it  will  use  to  group  the  input  data.  This  is  referred  to  as  self¬ 
organization.  Another  classification  of  types  of  neural  networks  lists  the  following  types 
(Anderson  and  McNeill,  1992): 

•  Networks  for  prediction-networks  that  attempt  to  make  projections  of  the  future. 

o  Feedforward,  Back  propagation, 
o  Delta  Bar  Delta, 
o  Extended  Delta  Bar  Delta, 
o  Directed  Random  Search, 
o  Functional  Link  Network, 
o  Self-organizing  map  into  back  propagation. 

•  Networks  for  classification- 

o  Learning  Vector  Quantization, 
o  Counter  Propagation  Network, 
o  Probabilistic  Neural  Network. 

•  Networks  for  data  association-recognition  of  occurrences  of  bad  data  that  can  span  all 
classifications. 

o  Hopfield  Network, 
o  Boltzmann  Machine, 
o  Hamming  Network, 
o  Bi-directional  Associative  Memory, 
o  Spatio  Temporal  Pattern  Recognition  (Avalanche). 

•  Networks  for  Data  Conceptualization-provide  the  ability  to  group  data  into 
classifications. 

o  Adaptive  Resonance  Network, 
o  Self-organizing  Map. 

•  Networks  for  data  filtering- 

o  Recirculation. 

NNS  models  may  not  have  a  way  to  determine  the  output  of  the  model  when  given  a 
certain  input  short  of  "running"  the  model  in  a  numerical  computer  simulation.  Thus, 
“experiments”  can  be  run  on  a  model,  in  ways  that  are  similar  in  some  respects  to  experiments 
run  on  human  subjects. 
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2.6.  Summary  of  Literature  Review  Topics. 


The  topics  in  this  literature  review  are  intended  to  provide  a  knowledge  understanding  of 
many  of  the  concepts  that  apply  to  the  COMPASS  paradigm  throughout  this  dissertation  to  those 
not  versed  in  U.S.  Army  operational  concepts,  concepts  in  human  factors  engineering  and 
research  psychology,  computer  simulation,  and  certain  multivariate  statistical  approaches.  The 
following  paragraphs  summarize  which  topics  apply  to  various  sections  of  this  dissertation. 

2.6.1.  Literature  Review  Topics  in  Phase  I. 

The  following  topics  from  the  literature  review  that  apply  to  discussions  in  Phase  I 
include: 

•  Anny  Operations  Orders. 

•  Anny  Battle  Command  System. 

•  Battlefield  Operating  Systems. 

•  Command  and  Control. 

•  TOC. 

•  MANPRINT. 

•  War. 

•  Evaluation  of  Human  Cognitive  Perfonnance. 

•  Ethnography  and  Naturalistic  Observation. 

•  Human  Computer  Interface. 

•  Critical  Incident  Reporting. 

•  Task  Analysis. 

•  Cognitive  Task  Analysis. 

•  Cluster  Analysis. 

•  Datamining  and  Neural  Networks. 

•  Sequential  Experimentation. 

2.6.2.  Literature  Review  Topics  in  Phase  II. 

The  following  topics  from  the  literature  review  that  apply  to  discussions  in  Phase  II 
include: 
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•  Expertise. 

•  Situation  Awareness. 

•  Stress. 

•  Decision  Making. 

•  Task  Perfonnance. 

•  Team  Performance. 

•  Workload. 

•  The  System 

•  Concept  of  the  Model. 

•  Computer  Simulation  Models. 

•  Task  Network  Modeling. 

•  Optimization  Techniques  Using  Computer  Simulation  Models. 

•  Simulations  of  Human  System  Performance. 

2.6.3.  Literature  Review  Topics  in  Discussion  Chapter. 

The  following  topics  from  the  literature  review  that  apply  to  discussions  in  the 
Discussion  chapter  include: 

•  Command  and  Control. 

•  TOC. 

•  Decision  Making. 

•  Task  Perfonnance. 

•  Team  Performance. 

•  Characteristics  of  Models. 
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3.  Phase  I  -  Ethnographic  Based  Naturalistically  Observed  Data  Collection  And  Neural 

Network  Analysis. 

The  overall  goal  of  this  research  is  to  develop  a  paradigm  of  experimental  tools  and 
methodologies  that  can  be  utilized  to  provide  an  evaluation  test  bed  for  the  examination  of  future 
systems  that  are  envisioned  to  be  deployed  into  a  TOC  environment.  Currently  this  is  not  known 
to  exist.  While  the  literature  reflects  an  extensive  amount  of  work  that  is  focused  on  the 
description  and  evaluation  of  the  military  C2  environment,  all  of  the  literature  surveyed  focuses 
on  the  human  and  team  performance  aspect  of  this  work  space,  (Cowings,  Toscano,  DeRoshia, 
and  Tauson,  1999;  Dryer,  1998;  Ford  et  al.,  1997;  Huron,  1997;  McGlynn  and  Pierce,  1997; 
Rasker,  Post,  and  Schraagen,  2000;  Reynolds,  1997),  to  name  a  small  representative  of  recent 
work.  This  study  goes  beyond  the  human  performance  aspect  of  the  C2S  and  is  intended  to 
consider  all  aspects  that  make  up  the  total  system  surrounding  the  C2  effort. 

3.1.  Method. 

This  data  collection  effort  is  expected  to  generate  large  amounts  of  raw  data  with  a 
hundred  or  more  IV  and  a  dozen  or  more  DV  through  hundreds  of  observation  cycles.  First, 
neural  network  and  datamining  procedures  are  used  to  identify  those  IV  that  are  suitable 
candidates  for  further  experimental  evaluation.  The  experimental  environment  that  will  be 
utilized  to  evaluate  these  IV  will  be  similar  to  previous  work  (Middlebrooks,  2001)  where  an 
existing  computer  simulation  of  the  TOC  system  provided  the  controlled  environment  whereby 
these  IV  can  be  evaluated. 

3.1.1.  Ethnographic  Based  Naturalistic  Observations  of  TOCs. 

This  phase  consisted  of  naturalistic  observations  and  data  collection  from  the  operation  of 
brigade  TOCs  during  simulated  combat  operations.  Direct  observation  of  activities  during 
critical  incidents  that  occur  during  the  combat  scenario,  augmented  by  impromptu  overheard  self 
reporting  during  critical  incident  interactions,  fonns  a  basis  for  the  collection  of  performance 
data.  This  is  especially  true  when  the  observer  is  accepted  and,  even  better,  ignored  while  in  the 
midst  of  the  group  being  observed  as  a  result  of  ethnographic  and  other  operator  interaction 
techniques.  Because  of  the  size  and  complexity  of  the  data,  procedures  were  developed  to 
evaluate  it  using  datamining  techniques  such  as  neural  networks  and  cluster  analysis.  The  data 
from  the  study  provided  information  on  159  IVs  and  75  DVs  with  164  hourly  observations.  In 
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the  future  this  dataset  will  grow  with  each  successive  TOC  observation.  A  benefit  of  the 
datamining  approach  is  that  the  data  can  grow  longitudinally  with  the  observations  from  each 
successive  exercise  adding  to  a  growing  data  store. 

3.1.2.  Data  Collection. 

In  order  to  evolve  the  ability  to  determine  those  things  that  are  of  significance  to  TOCs  in 
general  and  are  not  specific  to  any  one  TOC,  many  observations  over  time  are  required.  Some  of 
these  will  involve  longitudinal  observations  of  the  same  unit  through  successive  deployments  of 
their  TOC.  It  is  noted  that  Army  units  conduct  training  exercises  generally  for  one  of  three 
reasons.  First,  an  exercise  might  be  conducted  as  a  pure  training  event  where  the  unit 
commander  is  working  to  develop  and  maintain  the  warfighting  skills  of  the  unit.  Second,  the 
exercise  can  be  conducted  where  the  unit  is  being  evaluated.  Typically,  this  is  the  case  where  an 
external  cadre  of  evaluators  will  deploy  to  the  exercise  with  the  unit  and  observe  and  evaluate 
their  performance.  Third,  an  exercise  might  be  conducted  to  test  some  new  weapon  system, 
communications  system,  or  tactical  procedure  being  considered  for  fielding  to  the  general  Army 
population.  This  type  of  exercise  is  conducted  in  conjunction  with  a  test  and  evaluation 
organization,  and/or  the  new  system’s  developer  program  manager  to  determine  if  the  new 
system  performs  as  designed  and  required,  and  if  it  actually  improves  the  battlefield  performance 
capability  of  the  unit. 

The  data  collection  for  this  effort  was  able  to  observe  one  of  each  type  of  these  exercises. 
Exercise  Raider  Shadow,  conducted  in  April  2002,  was  a  test  exercise  evaluating  the  new 
Tactical  Unmanned  Aerial  Vehicle  (TUAV)  that  was  being  prepared  for  fielding  to  brigade 
TOCs.  Exercise  Ironhorse  Bonecrusher,  conducted  in  September  2002,  was  a  unit  conducted 
training  exercise  where  the  commander  conducted  the  exercise  according  to  his  own  agenda  and 
training  objectives.  Exercise  Warfighter,  conducted  in  November  2002,  was  an  evaluated 
exercise  where  a  team  from  Operations  Group  B  of  the  Battle  Command  Training  Program 
(BCTP)  from  Fort  Leavenworth,  Kansas  deployed  to  the  exercise  to  evaluate  the  battlefield 
proficiency  of  the  unit.  All  three  exercises  were  conducted  at  Fort  Hood,  Texas.  While  the 
research  in  this  dissertation  observed  three  TOC  deployments,  the  intent  is  to  develop  techniques 
and  procedures  that  will  form  the  basis  for  an  ongoing  effort.  As  the  data  store  grows  through 
successive  TOC  observations,  the  possibility  will  exist  that  the  simulations  will  be  able  to 
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relegate  to  the  noise  level  the  interpersonal  differences  between  deployments  of  individual  TOCs 
and  will  be  able  to  focus  on  those  core  issues  that  are  relevant  to  TOC  operations  in  general. 

Each  TOC  observation  begins  weeks  before  the  start  of  the  exercise  with  the  acquisition 
and  assimilation  of  information  that  describes  how  the  exercise  will  be  conducted  and  what  its 
tactical  objectives  will  be.  These  materials  include  OPORDs,  fragmentary  operations  orders 
(FRAGO)  as  they  are  developed  that  modify  the  original  OPORD,  tactical  map  overlays,  and 
intelligence  collection  plans,  for  example.  In  other  words,  all  of  the  tactical  materials  that  the 
exercise  unit  prepares  that  describe  how  it  intends  to  conduct  combat  operations  in  this  simulated 
training  scenario.  An  example  of  a  brigade  OPORD  is  at  Appendix  E  for  an  exercise  that  was 
conducted  in  April  2002  by  the  1st  Brigade,  4th  Infantry  Division  at  Fort  Hood,  Texas.  These 
OPORDs  are  supported  by  numerous  annexes  and  attachments,  however,  the  basic  document 
provides  the  reader  with  a  feel  for  the  level  of  detail  that  goes  into  the  planning  for  these  exercise 
events  which  mirror  what  is  anticipated  to  be  required  for  a  real  combat  operation. 

Once  the  exercise  began  the  TOC  was  observed  in  operation  by  a  data  gathering  effort 
designed  to  assimilate  as  much  information  as  is  possible  from  a  totally  non-interference  effort. 
This  included  observations  of  the  activities  of  the  personnel  in  the  TOC  along  with  their 
interactions  with  various  TOC  systems  such  as  the  ABCS.  It  also  incorporated  aspects  of 
cognitive  task  analysis  techniques  to  correlate  mental  processes  being  employed  to  the  tasks  and 
activities  being  attempted.  One  intent  during  this  process  was  not  take  any  action  or  make  any 
inquiry  of  TOC  personnel  that  could  potentially  cause  a  change  in  their  mode  of  operation  or 
what  their  span  of  focus  and  attention  is  at  any  one  time. 

The  ability  to  perform  naturalistic  observations  within  the  limited  space  available  in  a 
TOC  requires  equipment  that  not  only  is  capable  of  performing  the  observation  but  also  must  be 
highly  portable  with  self  contained  power  sources.  One  of  the  requirements  for  a  non-obtrusive 
presence  in  the  midst  of  the  human  operators  within  the  TOC  is  that  the  observer  must  make 
little  or  no  demands  for  resources  in  terms  of  power,  workspace,  furniture,  or  any  other  asset  that 
has  been  placed  within  the  TOC  for  use  in  conducting  the  mission.  To  be  sure,  it  takes  a 
considerable  effort  on  the  part  of  the  C2  team  to  set  up  the  workspace  and  populate  it  with  the 
tools  and  equipment  required  for  C2  operations.  Even  minor  considerations  such  as  sitting  down 
in  an  empty  chair  must  be  avoided  as  that  chair  was  carried  in  and  placed  there  for  the  use  of 
some  soldier  who  has  a  role  to  perform  in  the  functioning  of  the  TOC.  Along  with 
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considerations  for  ethnographic  role  associations  that  are  established  with  members  of  the  TOC 
team,  the  naturalistic  observer  team  member  must  be  totally  self  sufficient  within  the  workspace 
confines  in  order  to  blend  into  the  background  and  achieve  the  state  of  having  one’s  presence 
essentially  ignored. 

One  of  the  only  exceptions  to  this  philosophy  is  when  a  member  of  the  C2  team 
approaches  the  observer  with  inquiries  such  as  what  they  are  doing  there.  Following 
ethnographic  considerations  the  observer  should  treat  all  direct  contact  with  team  members  with 
equal  priority  regardless  of  whether  they  are  the  highest-ranking  commanding  officer  or  the 
lowest-ranking  radio  operator.  Once  the  individual  has  approached  the  observer  on  their  own 
volition  then  the  rules  of  non-interference  are  superseded  by  considerations  for  establishing 
rapport  with  the  team  member.  In  these  cases  if  a  chair  or  cup  of  coffee  is  offered  and  it  would 
be  impolite  to  refuse,  then  it  should  be  accepted.  These  conversations  and  encounters  often 
provide  a  rich  opportunity  to  fill  in  gaps  in  the  knowledge  of  how  the  TOC  operation  is 
organized  and  what  the  current  problems  and  priorities  for  the  C2  team  are.  However,  once  the 
encounter  has  run  its  logical  course  and  the  team  member  returns  to  his  or  her  duties  then  the 
observer  should  return  to  the  previous  non-interference  state  of  presence. 

Once  the  exercise  begins  observations  are  made  and  the  data  previously  described  are 
recorded  at  selected  intervals,  usually  hourly,  in  order  to  develop  a  longitudinal  database.  The 
philosophy  is  to  make  observations  that  can  be  recorded  in  a  quantitative  rather  than  qualitative 
manner.  The  further  desire  is  to  record  these  observations  using  automated  rather  than  manual 
techniques.  This  eliminates  any  requirement  for  manual  transcription  of  the  data  with  the 
resultant  time  delays,  probability  for  transcription  error,  and  requirement  for  labor  to  perform  the 
transcription.  It  also  provides  data  that  is  immediately  ready  for  reduction  from  the  raw  observed 
form  into  data  tables  and  analytical  evaluation  with  statistical  and  simulation  software  packages. 
To  accomplish  this  with  totally  portable  equipment  while  retaining  the  ability  for  the  observer  to 
be  able  to  move  around  in  the  TOC  requires  the  selection  of  pseudo-specialized  equipment  that 
meets  these  objectives. 

The  three  exercises  observed  during  this  study  were: 
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3. 1.2.1.  Exercise  Raider  Shadow,  1st  Brigade,  4th  Infantry  Division,  Fort  Hood,  Texas,  April 
2002. 

This  exercise  was  conducted  by  a  brigade  combat  team  (BCT)  and  was  performed  in 
conjunction  with  the  Operational  Test  Command  (OTC)  at  Fort  Hood  to  test  the  unmanned  aerial 
drone  called  the  Tactical  Unmanned  Aerial  Vehicle  (TUAV)  prior  to  its  initial  deployment  to 
brigades  across  the  Army.  The  TUAV  is  designed  to  provide  live  video  imagery  of  enemy 
forces  behind  the  line  of  contact  and  to  provide  the  ability  to  direct  artillery  fire  against  them.  A 
picture  of  the  TUAV  is  at  Figure  18  and  a  sample  of  its  video  display  is  at  Figure  19.  The 
exercise  was  conducted  using  the  wargame  simulation  JANUS  (not  an  acronym)  where  live 
friendly  and  enemy  forces  engage  each  other  on  an  electronic  battlefield.  The  algorithms  in 
JANUS  provide  the  ability  to  adjudicate  all  battlefield  losses  and  results.  Normally  this 
simulation  is  used  for  training  purposes  in  battlefield  command  and  control,  however,  this 
exercise  also  provided  the  ability  to  evaluate  the  new  TUAV  battlefield  asset  envisioned  to  be 
assigned  to  the  brigade  headquarters. 

During  the  course  of  the  two-week  exercise  there  were  74  hourly  observations  made  in 
the  TOC  63  independent  measures  and  6  dependent  measures.  Naturalistic  observation 
procedures  were  followed  that  employed  ethnographic  interaction  techniques  to  blend  into  the 
TOC  environment  so  that  observations  of  activity  occurring  within  the  TOC  could  be  conducted 
without  interfering  with  the  activity  being  observed. 


Figure  18  -  Tactical  Unmanned  Aerial  Vehicle  (TUAV) 
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Figure  19  -  Example  Video  Display  From  TUAV 

Before  the  start  of  the  exercise,  OPORDs  and  other  tactical  documents  was  collected  that 
provided  an  insight  as  to  how  the  unit  intended  to  react  to  the  tactical  scenario.  While  this 
exercise  was  conducted  on  a  24  hour  basis  only  about  eight  hours  in  the  day  could  be  observed 
because  of  the  size  of  the  observation  team,  which  was  one.  Other  demands  included  the  upload 
of  the  observed  data  at  the  end  of  the  day  to  the  main  simulation  computer  and  time  required  to 
recharge  batteries  and  care  for  observation  equipment.  During  the  course  of  the  exercise  careful 
attention  was  paid  to  ethnographic  considerations  for  interaction  with  the  members  of  the  TOC 
and  after  about  the  3ld  day  of  the  exercise  the  COMPASS  observer’s  presence  was  generally 
ignored  except  when  a  TOC  member  became  bored  and  looked  around  for  someone  to  talk  to. 

The  independent  measures  observed  fell  into  several  categories.  These  independent 
measures  provided  the  set  of  IV s  used  in  subsequent  analyses.  The  first  set  was  oriented  around 
the  first  six  of  seven  categories  of  Army  BOSs.  These  IV  and  the  BOS  they  relate  to  are  shown 
in  Table  8. 


142 


Table  8  -  TOC  Observation  IVs  by  BOS 
(Ford  et  al.,  1997) 


BOS 

Performance  Measure 

Intelligence 

1  Conduct  Intelligence  Planning 

2  Collect  Information 

3  Process  Information 

4  Disseminate  Intelligence 

Maneuver 

1  Conduct  Tactical  Movement 

2  Engage  Enemy  with  Direct  Fire  and  Maneuver 

Mobility  &  Survivability 

1  Overcome  Obstacles 

2  Enhance  Movement 

3  Provide  Countermobility 

4  Enhance  Physical  Protection 

5  Provide  Operations  Security 

6  Conduct  Deception 

7  Conduct  NBC  Defense 

Fire  Support 

1  Employ  Mortars 

2  Employ  Field  Artillery 

3  Employ  Close  Air  Support 

4  Conduct  Electronic  Collection  and  Attack 

5  Conduct  PSYOP 

6  Employ  Chemical  Weapons 

7  Conduct  Counter  Target  Acquisition  Operations 

8  Employ  Naval  Surface  Fires 

9  Coordinate,  Synchronize,  and  Integrate  Fire  Support 

Air  Defense 

1  Take  Active  Air  Defense  Measures 

2  Take  Passive  Air  Defense  Measures 

Command  &  Control 

1  Plan  for  Combat  Operations 

2  Direct  &  Lead  Unit  During  the  Preparation  Phase  of  the  Battle 

3  Direct  &  Lead  Unit  in  Execution  of  Battle 

The  second  set  of  observation  IVs  evaluated  the  use  of  the  various  factors  including 
communications  systems,  intelligence  collection  assets,  the  presence  or  absence  of  key 
personnel,  environmental  conditions,  and  certain  miscellaneous  factors  concerning  the  operation 
of  the  TOC.  Table  9  lists  these  systems. 
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Table  9  -  TOC  Observation  I  Vs  by  Other  Categories 


Category 

Performance  Measure 

Communications  Usage 

-  AFATDS  -  Advanced  Field  Artillery  Tactical  Data  System 

-  AMPS  -  Aviation  Mission  Planning  System 

-  Applique 

-  ASAS  -  All  Source  Analysis  System 

-  CTIS  -  The  Combat  Terrain  information  System 

-  CSSCS  -  The  Combat  Service  Support  and  Control  System 

-  FAADC2  -  Forward  Area  Air  Defense  Command  and  Control 

-  FBCB2  -  Force  XXI  Battle  Command  Brigade  and  Below 

-  IMETS  -  Integrated  Meteorological  System 

-  MCS/P  -  Maneuver  Control  Systems/Phoenix 

-  AMDWS.  ADA 

-  CGS.  JSTARS 

Intelligence  Collection  Usage 

-  TUAV  -  Tactical  Unmanned  Aerial  Vehicle 

-  JSTARS  -  Joint  &  Strategic  Tactical  Airborne  Radar  System 

-  GSR  -  Ground  Surveillance  Radar 

Key  Personnel  Present  or  Absent 

-  Commanding  Officer 

-  Executive  Officer 

-  Battle  Captain 
-S3 

-  S3  RTO 

-  S2 

-  FSO 

-  Engineer 

-  ALO 

Environmental  Conditions 

-  Sky  Condition:  Clear- 1;  Overcast-2;  Rain-3 

-  Wind  Condition:  0-  none;  1-  low;  2-  moderate;  3-  high 

-  Light  Level,  Foot  Lamberts  -  Background  at  20  ft. 

-  Light  Level,  Foot  Lamberts  -  Map  Display  (Rear  Projection)  at  20  ft. 

-  Noise,  dbA 

-  Dry  Bulb  Temperature,  degrees  F 

-  Wet  Bulb  Temperature,  degrees  F 

-  Relative  Humidity,  % 

-  Hour  of  the  Day 

-  Day  of  the  Exercise 

Miscellaneous  Factors 

-  Battle  Timing 

-  Battle  Tempo 

-  Reconnaissance  Operations 

-  Information  Operations 

-  Tactics 

-  Observed  Activity  /  Stress  Level  in  TOC 

The  dependent  measures  that  were  observed  are  those  activities  that  the  TOC  is  trying  to 
perform  at  the  system  level.  Generally,  these  measures  are  the  mission  oriented  tactical  activities 
or  their  components  whose  activity  the  TOC  is  trying  to  direct  on  the  battlefield.  These 
observations  were  made  first  simply  as  what  activity  was  being  attempted  when.  They  were 
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recorded  in  the  three  categories  of  primary  task  perfonnance,  secondary  task  performance,  and 
tiertiary  task  perfonnance  as  being  representative  of  the  fact  that  the  TOC  work  group  will 
usually  be  performing  more  than  one  activity  at  a  time.  The  first,  second,  and  third  perfonnance 
tasks  are  observed  as  those  goal  oriented  activities  in  progress  and  their  relative  priority.  These 
dependent  measures  become  the  DVs  in  the  subsequent  analyses.  Table  10  lists  these  dependent 
measures.  At  the  end  of  the  exercise  when  the  TOC  performance  is  rated  by  either  the  training 
group  themselves  or  by  external  evaluators  in  the  After  Action  Review  (AAR)  meeting,  the 
ratings  are  correlated  back  to  the  observations  made  during  the  exercise  to  indicate  a  measure  of 
success  for  the  TOC  system  in  the  performance  of  its  mission  activities  as  indicated  by  a 
percentile  rating  for  the  appropriate  DV. 

Table  10  -  TOC  Dependent  Measures  of  Perfonnance 


Category 

Performance  Measure 

TOC  Performance  Task 

Dependent  Measures. 

-1  Departure  from  the  assembly  area. 

-2  Passage  of  lines. 

-3  Movement  to  the  line  of  departure. 

-4  Breach  of  main  obstacle  belt. 

-5  Penetration  of  defensive  positions. 

-6  Reaction  to  counterattack  forces. 

-7  River  crossing. 

-8  Seizure  of  key  terrain. 

-9  Seizure  of  objective. 

-10  Destruction,  capture,  or  bypass  of  enemy  force. 

-1 1  Fixing  enemy  in  position. 

-12  Synchronization  with  supporting  forces. 

-13  Use  of  reserves. 

-14  Deep  operations. 

-15  Destruction  of  first  echelon  forces. 

-16  Destruction  of  follow-on  forces. 

-17  Commitment  of  counterattack  forces. 

-18  Deception  activities. 

-19  Rear  operations. 

-20  Entry  into  area  of  operations. 

-21  Peacekeeping  operations. 

-22  Transfer  of  mission. 

These  preliminary  data  contained  elements  that  were  categorical,  binary,  and  interval  in 
nature  and  were  coded  into  a  spreadsheet  for  field  observation.  An  excerpt  of  the  final  data 
collection  spreadsheet  is  shown  in  Figure  20. 
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Central  Standard  Time  (CST)  - 


Hour  Of  The  Day: 
Day  Of  The  Exercise: 


ET  EU  EV 


CD  CD  CD 
CD  CD  O 

c~-  oo  os 
CN  n  fN 
CD  CD  CD 
CD  CD  CD 

C2  C?  C3 

Os  Os  O. 

§  §  § 


8  9 
7  7 


Primary  Task  Select  Task  Number  From  List:|1  Departure  from  the  assembly  area.|2  F 

Secondary  Task  Select  Task  Number  From  List:|1  Departure  from  the  assembly  area.|2  F 

Ancillary  Task  Select  Task  Number  From  List:|1  Departure  from  the  assembly  area.|2  F 


Battlefield  Operating  Systems  (BOS) 


Intelligence 
Maneuver 
Fire  Support 
Mobility  and  Survivability 
Air  Defense 
Command  and  Control 

Communications 

Rate  each  system 
on  a  scale  of  1  (low) 
to  10  (high)  as  to  its 
level  of  observed  usage 


Battlefield  Function  (BF) 
Battlefield  Function  (BF) 
Battlefield  Function  (BF) 
Battlefield  Function  (BF) 
Battlefield  Function  (BF) 
Battlefield  Function  (BF) 


llndicate  Which  BF  Is  Being 
llndicate  Which  BF  Is  Being 
llndicate  Which  BF  Is  Being 
llndicate  Which  BF  Is  Being 
llndicate  Which  BF  Is  Being 
llndicate  Which  BF  Is  Being 


Performedll  Con 
Performedll  Co 
Performedll  Emi 
Performedll  Ovi 
Performed^  Tali 
Performedll  Plaj 


Intelligence  Collection 


Rate  usage  of  Each  System:  1-10, 10  Highest 
AFATDS  -  Advanced  Field  Artillery  Tactical  Data  System 
AMPS  -  Aviation  Mission  Planning  System 
Applique 

ASAS  -  All  Source  Analysis  System 

CTIS  -  The  Combat  Terrain  Information  System 

CSSCS  -  The  Combat  Service  Support  and  Control  System 

FAADC2  -  Forward  Area  Air  Defense  Command  and  Control 

FBCB2  -  Force  XXI  Battle  Command  Brigade  and  Below 

IMETS  -  Integrated  Meteorological  System 

MCSfP  -  Maneuver  Control  SystemsfPhoenix 

AMDWS.  ADA 

CGS.  JSTARS 

Rate  usage  of  Each  System:  1-10. 10  Highest 
TUAV  -  Tactical  Unmanned  Aerial  Vehicle 
JSTARS  -  Joint  6c  Strategic  Tactical  Airborne  Radar  System 
GSR  -  Ground  Surveillance  Radar 


1  1 


Figure  20  -  Excerpt  of  Raw  Data  Collection  Spreadsheet 

3. 1.2.2.  Exercise  Ironhorse  Bonecrusher,  1st  BCT,  1st  Cavalry  Div.,  Fort  Hood,  Texas,  Sept., 
2002. 

This  exercise  was  an  internal  training  event  that  was  conducted  as  the  unit  prepared  itself 
for  deployment  to  the  National  Training  Center  (NTC)  at  Fort  Irwin,  California.  An  NTC 
deployment  is  the  only  time  a  brigade  sized  unit  undergoes  full  scale  field  maneuvers  against  a 
live  opposing  force  playing  the  enemy.  The  NTC  exercise  rotation  provides  a  full  up  evaluation 
of  the  unit’s  warfighting  capability  and  will  only  occur  once  during  the  tour  of  duty  for  a 
commander  and  his  staff.  This  is  the  most  intensive  tactical  field  event  that  a  unit  will  undergo  in 
peacetime  and,  as  a  result,  an  intensive  amount  of  preparation  and  training  is  conducted  to 
prepare  for  it.  Exercise  Ironhorse  Bonecrusher  was  just  such  a  preparatory  training  event.  It 
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represents  one  of  the  few  times  at  Fort  Hood  that  a  brigade  sized  unit  will  conduct  live  field 
training  of  the  entire  brigade  as  opposed  to  simulations  based  training. 

During  the  course  of  the  two-week  exercise  there  were  44  hourly  observations  made  in 
the  TOC  for  159  independent  measures  and  75  dependent  measures.  Naturalistic  observation 
procedures  were  again  followed  that  employed  ethnographic  interaction  techniques  to  blend  into 
the  TOC  environment  so  that  observations  of  activity  occurring  within  the  TOC  could  be 
conducted  without  interfering  with  the  activity  being  observed.  Data  was  collected  and  merged 
into  the  previous  dataset. 

3. 1.2. 3.  Exercise  Warfighter,  1st  Brigade,  4th  Infantry  Division,  Fort  Hood,  Texas,  November 
2002. 

The  division  level  equivalent  to  the  brigade’s  NTC  rotation  is  called  the  Battle  Command 
Training  Program  (BCTP).  Instead  of  the  unit  deploying  to  the  exercise  location  as  with  the 
NTC  rotation,  a  BCTP  team  from  Fort  Leavenworth,  Kansas  deploys  to  the  division’s  location 
and  conducts  an  evaluation  exercise  in  computer  simulation.  While  the  division  commander  and 
staff  are  conducting  the  exercise  on  the  simulation’s  electronic  battlefield,  all  three  brigade 
headquarters  TOCs  from  the  division  deploy  to  the  field  to  support  the  exercise.  The 
organization  of  the  exercise  is  that  a  cadre  from  each  battalion  role  plays  the  activities  of  their 
battalion  on  the  computer  simulation  terminals  and  report  activities  and  receive  command 
guidance  from  the  brigade  TOC  who  further  coordinates  with  the  division  TOC.  Both  the 
division  and  all  three  brigade  TOCs  are  fully  deployed  to  the  field  and  if  they  did  not  know  that 
the  battle  was  actually  being  played  out  in  simulation  they  might  think  the  battle  was  in  actual 
progress.  This  is  because  at  the  TOCs  in  the  field  the  commanders  and  staffs  are  receiving 
communications  from  role-playing  subordinates  and  actual  adjacent  and  higher  headquarters 
command  centers. 

This  observation  was  with  the  first  brigade  of  the  division  while  it  was  deployed  during 
one  of  these  exercises.  During  the  course  of  the  two-week  exercise  there  were  46  hourly 
observations  made  in  the  TOC  for  159  independent  measures  and  75  dependent  measures. 
Naturalistic  observation  procedures  were  again  followed  that  employed  ethnographic  interaction 
techniques  to  blend  into  the  TOC  environment  so  that  observations  of  activity  occurring  within 
the  TOC  could  be  conducted  without  interfering  with  the  activity  being  observed.  Data  was 
collected  and  merged  into  the  previous  dataset. 
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3.1.3.  JNNS  Simulation. 


The  neural  network  simulation  used  in  this  study  is  called  Java  Neural  Network 
Simulation  (JNNS)  developed  at  the  University  of  Tubingen,  Wilhelm-Schickard  Institute  in 
Tubingen,  Gennany  (Fischer  et  ah,  2001).  JNNS  runs  in  a  Microsoft  Windows  environment  and 
is  a  derivative  and  subset  of  a  more  complete  neural  network  simulation  called  Stuttgart  Neural 
Network  Simulation  (SNNS)  that  executes  on  Unix  based  computers.  This  simulation  allows  the 
development  of  neural  networks  that  can  be  trained  and  pruned  to  reveal  relationships  to  be 
identified  as  to  the  significance  of  input  nodes  (corresponding  to  IV)  and  output  nodes 
(corresponding  to  DV).  It  supports  many  different  learning  algorithms  and  architectures 
including  the  feed-forward,  back-propagation  learning  algorithm  and  multi-layer  perceptron 
architecture  used  in  this  dissertation. 

For  this  study  the  JNNS  was  configured  in  a  multi-layer  perceptron  (MLP)  architecture 
with  a  feed-forward,  back-propagation  learning  algorithm.  There  where  three  layers  in  the 
network.  Input  and  output  layers  represented  the  independent  (input)  variables,  dependent 
(output  layer)  variables  along  with  one  hidden  layer  of  30  nodes.  Utilizing  the  infonnation 
collected  from  the  exercise  observations,  the  independent  measures  provided  data  for  159  nodes 
in  the  input  layer  and  the  dependent  measures  provided  data  for  75  nodes  in  the  output  layer. 

The  number  of  hidden  layers  and  the  number  of  nodes  in  each  hidden  layer  is  determined  by 
subjective  opinion  based  upon  the  following  general  guidelines: 

•  There  should  be  as  few  hidden  layers  as  possible,  however,  for  MLP  architectures 
there  must  be  at  least  one.  If  successful  training  of  the  network  cannot  be 
achieved  with  one  hidden  layer  then  increase  the  number  of  layers  one  at  a  time 
until  full  training  can  be  accomplished. 

•  The  number  of  nodes  in  each  hidden  layer  should  be  as  small  as  possible  to  create 
an  efficient  network.  The  general  guideline  is  to  start  with  approximately  half  the 
number  of  input  nodes.  If  successful  training  of  the  network  cannot  be  achieved 
with  this  number  then  make  successive  runs  with  ever-greater  numbers  of  hidden 
nodes  until  training  can  be  achieved. 

In  this  case  the  number  of  hidden  layers  was  selected  as  one,  and  the  number  of  nodes  in  this 
layer  as  90.  Successful  training  was  achieved  at  approximately  350  epochs  with  no  over 
training.  Successful  pruning  was  achieved  at  100  cycles  and  the  pruned  results  were  used  to 
factor  screen  the  input  parameters  to  set  up  post  processing  of  the  results.  The  remaining  IV 
after  pruning  were  used  to  generate  mathematical  models  using  traditional  linear  regression 


148 


analysis.  The  DV  to  be  modeled  were  selected  as  those  dependent  measures  that  had  at  least  10 
observations  in  the  dataset. 

The  checklist  procedure  used  to  configure  and  run  JNNS  for  this  dataset  is  at  Appendix  F. 
A  general  description  of  the  process  and  the  results  is  in  the  following  paragraphs: 

3 . 1 .3 . 1 .  Designing  the  JNNS  Network  Architecture. 

JNNS  uses  a  graphical  user  interface  (GUI)  to  lay  out  and  design  neural  networks  for 
analysis.  The  first  step  is  to  place  all  of  the  input,  hidden,  and  output  nodes  into  a  design  pallet 
as  shown  in  Figure  2 1 .  There  is  one  input  node  for  each  independent  measure  in  either  binary  or 
standardized  (0,1)  form,  a  set  number  of  hidden  nodes  according  to  the  network  design 
guidelines  previously  discussed,  and  one  output  node  for  each  dependent  measure  in  either 
binary  or  standardized  (0,1)  form. 
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Figure  21  -  NNS  Node  Identification  and  Network  Layout 
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The  next  step  is  to  establish  the  linkages  between  the  nodes  in  each  layer.  While  in  a 
biological  neural  network  many  nodes  or  neurons  are  connected  to  many  other  neurons,  in  an 
artificial  neural  network  each  input  node  is  connected  to  each  hidden  layer  node  that  is  then 
forward  connected  to  each  output  node.  The  similarity  between  biological  and  artificial  neural 
networks  and  nodes  is  illustrated  in  Figure  22. 


Input  Layer  Hidden  Layer  Output  Layer 


Figure  22  -  Comparison  of  Biological  and  Artificial  Neural  Networks 

(Kwahk,  2002) 

(Used  With  Permission  of  the  Author) 

While  each  neuron  performs  a  component  of  the  thought  process,  each  artificial  node  translates 
the  input  value  weight  from  the  link  connecting  it  to  another  node  according  to  a  predetermined 
activation  or  transfer  function.  Possible  activation  functions  for  artificial  neural  networks  is 
shown  in  Figure  23. 
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Identity  Step 


Ramping 


Figure  23  -  Artificial  Neural  Network  Activation  Functions 
(Kwahk,  2002) 

(Used  With  Permission  of  the  Author) 

When  the  command  to  establish  all  the  interconnecting  links  is  submitted  to  JNNS  it 
connects  each  input  node  to  each  hidden  node  and  forward  connects  each  hidden  node  to  each 
output  node.  Each  link  value  is  then  initialized  with  a  random  number  as  a  starting  value  to 
begin  the  process  of  training  where  each  training  cycle  recalculates  each  link  value  according  to 
the  activation  function  being  performed  by  the  nodes.  The  initialized  preliminary  data  neural 
network  that  is  ready  for  training  with  JNNS  is  shown  at  Figure  24.  This  and  the  following 
neural  network  diagram  figures  presented  later  are  acknowledged  to  be  shown  at  a  scale  that 
makes  them  unreadable,  however,  they  are  shown  only  to  provide  a  graphical  view  of  the 
complexity  of  the  networks  used  in  this  research. 
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Figure  24  -  JNNS  TOC  Network  After  Link  Creation  and  Initialization 


3. 1.3.2.  Data  Preparation. 

Preparation  of  the  exercise  data  for  analysis  follows  the  JNNS  requirement  that  the  input 
data  streams  be  organized  into  patterns  of  binary  or  standardized  information.  Allowable  data 
types  are  binary  (0,1)  and  standardized  in  the  range  of-1  to  +  1  or  0  to  +1 .  The  binary  data 
could  be  transcribed  and  used  in  its  original  form.  The  interval  data  such  as  %  humidity  was 
standardized  into  a  range  of  0  to  +1.  The  categorical  data  was  separated  into  a  subset  of  binary 
variables.  For  example,  the  variable  “Primary  Task”,  had  categorical  values  as  shown  in  Table 
11. 
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Table  1 1  -  Selection  Values  For  The  Variable  “Primary  Task” 


Select  Task  Number  From  List: 

1  Departure  from  the  assembly  area. 

2  Passage  of  lines. 

3  Movement  to  the  line  of  departure. 

4  Breach  of  main  obstacle  belt. 

5  Penetration  of  defensive  positions. 

6  Reaction  to  counterattack  forces. 

7  River  crossing. 

8  Seizure  of  key  terrain. 

9  Seizure  of  objective. 

10  Destruction,  capture,  or  bypass  of  enemy  force. 

1 1  Fixing  enemy  in  position. 

12  Synchronization  with  supporting  forces. 

13  Use  of  reserves. 

14  Deep  operations. 

15  Destruction  of  first  echelon  forces. 

16  Destruction  of  follow-on  forces. 

17  Commitment  of  counterattack  forces. 

18  Deception  activities. 

19  Rear  operations. 

20  Entry  into  area  of  operations. 

21  Peacekeeping  operations. 

22  Transfer  of  mission. 


Figure  20,  shown  previously,  shows  that  selections  of  1 1,  12,  and  21  were  made  from  this 
selection  list  during  collection  of  the  data.  Therefore,  three  new  binary  variables  were  created 
called  PrimaryTaskl  1,  Primary_Task_12,  and  Primary_Task_21  were  created  and  assigned 
values  of  0  or  1  depending  on  whether  or  not  they  were  selected  during  the  observation  period. 
Likewise,  new  binary  variables  Dayl  to  Day  10  and  Hourl  to  Hour24  were  created  to  represent 


153 


the  day  of  the  exercise  and  hour  of  the  day  that  the  observation  was  made.  In  JNNS  terminology 
each  hourly  data  collection  became  a  pattern  of  data  for  each  observation.  An  excerpt  of  this 
transcribed  preliminary  data  is  shown  in  Figure  25. 
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Figure  25  -  Transcribed  Preliminary  Data 


Text  strings  were  then  added  to  the  spreadsheet  to  set  up  the  required  data  text  fonnats  for  JNNS 
and  the  data  was  transposed  to  make  the  hourly  collection  patterns  appear  horizontally.  A 
sample  of  this  transposed  data  is  at  Figure  26. 
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Figure  26  -  Transposed  Preliminary  Data 


The  data  were  then  saved  as  a  pure  text  file  with  space  delimiters  between  each  cell  from  the 
spreadsheet.  Appropriate  JNNS  header  markings  were  added,  the  file  extension  was  changed  to 
.PAT  and  the  file  was  ready  to  be  read  by  the  JNNS  simulation.  An  excerpt  of  the  final  pattern 
file  is  at  Figure  27. 
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SNNS  pattern  definition  file  V3.2 
generated  at  Fri  Feb  22  15:58:23  2002 


No.  of  patterns  :  74 
No.  of  input  units  :  29 
No.  of  output  units  :  6 

#•  input  pattern  1  : 

1000000000100000000000010001010  0. 0  0. 1  0. 1  C 

#  output  pattern  1  : 

0  1  0  0  0  0 

#  input  pattern  2  : 

1000000000010000000000010001010  0. 0  0. 1  0. 1  C 

#  Output  pattern  2  : 

0  1  0  0  0  0 

#  input  pattern  3  : 

1000000000001000000000010001100  0. 1  0. 1  0. 1  C 

#  output  pattern  3  : 

0  1  0  0  0  0 

#  input  pattern  4  : 

1000000000000100000000010001010  0. 1  0. 2  0. 1  C 

#  output  pattern  4  : 

0  1  0  0  0  0 

#  input  pattern  5  : 

100  0(  000000000010000000010001010  0.2  0.2  0. 1  C 

#  output  pattern  5  : 

0  1  0  0  0  0 

it  Trim  it  nattarn  f>  • 

Figure  27  -  JNNS  Preliminary  Pattern  Data  File 


3.1.4.  Apparatus. 


The  equipment  employed  for  this  research,  although  highly  specialized,  consists  entirely 
of  off-the-shelf  instrumentation,  computer  hardware,  and  software  programs.  Field  equipment, 
including  the  tablet  computer,  was  procured  from  commercial  sources  and  each  unit  met  the 
requirement  to  operate  with  self  contained  power  and  be  completely  portable.  The  analysis 
computer  was  custom  built  using  commercially  available  components  with  the  design  goal  to 
configure  a  hardware  and  software  system  with  then  state  of  the  art  capabilities  in  terms  of 
processor  speed,  data  throughput,  and  storage  capability.  The  JNNS  simulation  was  available  as 
freeware  from  its  developers.  The  Micro  Saint™  discrete  event  simulation  system,  the  CoHOST 
TOC  simulation,  and  all  of  the  equipment  used  was  provided  by  funding  made  available  by  the 
U.S.  Army  Research  Laboratory. 


3. 1.4.1.  Data  Collection. 


The  naturalistic  observations  were  made  at  hourly  time  intervals  and  included  all 
activities  that  could  be  observed  and  recorded  in  the  TOC  regardless  of  how  seemingly 
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insignificant  the  observation  might  be.  These  recorded  data  included  information  regarding  the 
current  tactical  mission,  the  battlefield  operating  systems  (BOS)  in  current  employment, 
communications  systems  usage,  intelligence  collection  efforts  in  progress,  types  of  activity  and 
relative  stress  levels  in  the  work  group,  which  key  team  members  are  present  at  the  time  of  the 
observation,  and  observable  environmental  factors.  The  equipment  to  perform  these 
observations  must  have  self-contained  power  and  be  highly  portable.  This  equipment  included: 

•  A  tablet  computer  running  a  standard  Microsoft  Windows  operating  system  and 
Excel  spreadsheet  that  was  used  to  record  the  observations  which  were  made  on 
quantitative  scales  set  up  for  that  purpose.  The  computer  was  a  Fujitsu  model 
Point  510  running  Windows  95  with  Microsoft  Excel  installed  and  is  illustrated  in 
Figure  28.  It  is  a  compact  unit  with  a  touch  screen  display  integrated  into  the 
body  of  the  unit.  A  stylus  is  used  to  enter  data  on  the  touch  screen  and 
handwriting  recognition  software  translates  the  handwritten  entry  into  computer 
text  for  entry  into  the  data  file. 


Figure  28  -  Fujitsu  Touch  Screen  Tablet  Computer 

The  data  file  was  set  up  in  an  Excel  spreadsheet.  A  full  listing  of  the  spreadsheet 
for  the  first  few  hours  of  data  collection  is  at  Appendix  A. 

•  A  light  meter  for  recording  ambient  and  display  light  levels  in  the  TOC.  Figure 
29  shows  a  Spectra  Mini-Spot  Silicon  Cell  Spotmeter  by  Photo  Research  / 
Kollmorgen  Corporation.  This  meter  provides  a  spot  reading  in  foot-lamberts  of 
the  targeted  surface. 
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Figure  29  -  Spectra  Mini-Spot  Lightmeter 

•  A  sound  meter  for  recording  the  ambient  noise  level  within  the  TOC.  Figure  30 
shows  a  Quest  Electronics  model  215  Sound  Level  Meter  capable  of  recording 
noise  levels  in  decibels  in  either  the  A,  B,  or  C  weighted  scales  in  Db  ranges  of- 
30  to  +140.  The  A  weighted  scale  will  be  utilized  as  it  most  closely  matches  the 
frequency  response  pattern  of  the  human  ear  (Casali,  2001). 
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Figure  30  -  Quest  Electronics  Model  215  Sound  Level  Meter 

•  A  Psychrometer  for  measuring  wet  and  dry  bulb  temperature  from  which  relative 
humidity  can  be  detennined.  Figure  3 1  shows  a  Vista  Scientific  Corporation 
psychrometer  with  identical  mercury  thermometers.  A  fabric  sock  covers  one  of 
the  thennometer  bulbs.  The  unit  has  a  built  in  battery  operated  fan  that  cools  the 
sock  covered  bulb  when  it  is  moistened  to  provide  the  wet  bulb  reading.  A  slide 
rule  scale  on  the  side  of  the  unit  allows  the  determination  of  the  relative  humidity 
from  the  recorded  wet  and  dry  bulb  readings. 


Figure  31  -  Vista  Scientific  Corporation  Battery  Operated  Psychrometer 
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3. 1.4.2.  Computer  Simulations. 


Because  of  the  potentially  large  databases  and  number  of  required  simulation  iterations 
anticipated  for  the  analysis,  a  state  of  the  art  Microsoft  Windows™  based  dual  processor 
computer  was  configured  to  support  the  effort.  The  specifications  for  this  computer  system  are: 

•  Dual  Pentium  IV  Xeon  processors  each  with  a  clock  speed  of  1.8 
GigaHertz  (GHZ). 

•  Bus  Rate:  400  MegaHertz  (MHZ). 

•  Memory:  1  GigaBytes  (GB)  of  DDR  high  speed  memory. 

•  Disk  storage:  180  GB  in  a  striped  RAID  array. 

•  Operating  System  -  Windows  2000  Professional™. 

•  Dual  21”  Monitors  each  with  a  screen  area  of  1600  x  1200  pixels  at  a 
16  bit  color  depth. 


3.2.  Results. 

The  results  of  this  Phase  I  study  are  mathematical  models  that  quantitatively  describe  the 
relationship  of  observed  independent  performance  measures  in  the  TOC  against  the  mission 
based  dependent  performance  measures  of  what  activities  the  TOC  was  trying  to  accomplish  or 
control  on  the  battlefield.  Deviation  of  these  models  includes  the  training  and  pruning  of  the 
neural  network  followed  by  generation  of  the  models  through  traditional  linear  regression 
procedures. 

3.2.1.  JNNS  Network  Training. 

In  neural  network  terminology,  the  tenn  “training”  is  used  to  describe  the  NNS  state 
where  it  has  programmed  itself  to  replicate  the  set  of  output  conditions  that  were  observed  for 
each  input  condition  in  each  of  the  observations  in  the  dataset.  Each  observation  is  called  a 
“pattern”  of  data.  Thus,  the  164  hourly  observations  from  the  3  exercises  that  were  observed 
become  164  patterns  of  input  and  output  data. 

The  determination  of  when  a  network  has  achieved  full  training  is  a  subjective  one.  If  the 
training  process  is  stopped  before  a  100%  training  state  has  been  achieved  then  the  network 
outputs  do  not  completely  represent  the  observed  output  states  for  each  input  state.  If  the 
training  process  is  allowed  to  continue  beyond  a  100%  training  state  then  the  possibility  exists 


160 


that  the  network  will  over  train  itself  and  start  representing  network  noise.  The  actual 
identification  of  a  fully  trained  network  is  when  the  mean  square  of  the  error  of  the  outputs  is  at  a 
minimum. 

JNNS  provides  an  error  curve  display  to  assist  the  analyst  in  identifying  the  number  of 
epochs  to  allow  the  network  to  run  to  achieve  full  training.  To  effectively  utilize  the  error  curve, 
a  portion  of  the  dataset  is  separated  from  the  rest  and  is  used  as  a  validation  dataset. 
Approximately  25%  of  the  original  data  is  typically  used  for  this  purpose.  For  this  study,  as  a 
result,  25%  of  the  data  was  randomly  selected  from  the  original  training  dataset  and  was 
separated  into  a  validation  dataset.  During  training  JNNS  displays  the  error  curves  from  both  the 
training  and  validation  datasets  for  analysis.  If  the  validation  curve  flattens  to  a  horizontal  line 
across  successive  epochs  then  it  is  interpreted  to  mean  that  the  network  does  not  go  into  an  over 
training  state  regardless  of  the  number  of  epochs.  If  the  validation  curve  starts  to  increase  after 
an  initial  decrease,  then  the  point  of  the  increase  indicates  the  number  of  epochs  required  for  full 
training  and  that  should  not  be  exceeded  to  prevent  over  training.  An  example  of  this 
relationship  is  shown  in  an  error  curve  display  in  Figure  32. 


Figure  32  -  Example  Error  Graph  for  a  JNNS  Neural  Network 
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Ex.  1-3  Combined  Data  Learning  Run  #3  -  1000  Epochs 


Figure  33  -  TOC  JNNS  Training  Network  Diagram  With  Error  Curve. 

Following  completion  of  the  training  cycle  the  network  is  declared  to  be  fully  trained  and  ready 
for  further  analysis.  However,  to  validate  the  results  of  the  network  training  run,  a  correlation 
analysis  can  be  performed  comparing  the  output  node  values  of  the  trained  network  to  the  values 
in  the  original  dataset  that  show  the  field  observations.  A  sample  of  the  output  DV  values  as 


For  this  network  both  curves,  as  shown  in  Figure  33,  approach  a  flat  line  over  the  number 
of  epochs  reflecting  that  the  network  is  stable  and  will  not  attempt  to  over  train  itself.  The  dark 
curve  is  the  error  curve  from  the  training  dataset  and  the  light  line  is  from  the  validation  dataset. 
These  curves  indicate  that  full  training  was  achieved  at  approximately  550  epochs  for  this  test 
and  that  additional  epochs  will  not  cause  over  training. 
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they  were  observed  in  the  TOCs  is  at  Table  12,  and  a  sample  of  the  output  DV  values  from  the 
trained  JNNS  TOC  simulation  is  at  Table  13.  For  reference,  the  input  and  trained  output  DV 
values  from  a  small  study  are  presented  at  Appendix  G.  Casual  inspection  of  Tables  12  and  13 
show  what  appears  to  be  closely  matching  data  where  the  trained  values  approach  either  “0”  or 
“1”  in  patterns  that  match  the  binary  values  in  the  observation  data.  However,  trying  to  manually 
compare  two  tables  of  75  columns  representing  the  DVs,  and  139  rows  representing  the  139 
observation  patterns  is  not  possible.  A  simple  correlation  analysis  will  verify  if  this  relationship 
is  consistent  for  all  the  trained  patterns  thereby  validating  the  accuracy  of  the  trained  status  of  the 
JNNS  simulation. 

Table  12  -  Sample  of  Output  DV  Values  From  the  TOC  Observation  Data. 


Observed  DV  Values 


ITP1 

ITP2 

ITP3 

ITP4 

ITP5 

ITP6 

ITP7 

ITP8 

ITP9 

ITP10 

#  Output  Pattern  1: 

0 

0 

0 

0 

0 

0 

0 

0 

0 

1 

#  Output  Pattern  2: 

0 

0 

0 

0 

0 

0 

0 

0 

0 

1 

#  Output  Pattern  3: 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

#  Output  Pattern  4: 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

#  Output  Pattern  5: 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

#  Output  Pattern  6: 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

#  Output  Pattern  7: 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

#  Output  Pattern  8: 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

#  Output  Pattern  9: 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

#  Output  Pattern  10: 

0 

0 

1 

0 

1 

0 

0 

0 

0 

0 

#  Output  Pattern  11: 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

#  Output  Pattern  12: 

0 

0 

1 

0 

0 

0 

0 

0 

0 

0 

#  Output  Pattern  13: 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

#  Output  Pattern  14: 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

#  Output  Pattern  15: 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 
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Table  13  -  Sample  of  Output  DV  Values  From  the  JNNS  Trained  Data. 


Trained  JNNS  DV  Values 


OTP1 

OTP2 

OTP3 

OTP4 

OTP5 

OTP6 

OTP7 

OTP8 

OTP9 

OTP10 

#1.1 

0.008 

0.014 

0.025 

0.012 

0.036 

0.000 

0.031 

0.020 

0.013 

0.984 

#2.1 

0.005 

0.040 

0.029 

0.002 

0.027 

0.002 

0.016 

0.013 

0.002 

0.989 

#3.1 

0.002 

0.003 

0.010 

0.002 

0.012 

0.001 

0.019 

0.004 

0.017 

0.004 

#4.1 

0.004 

0.000 

0.006 

0.000 

0.000 

0.002 

0.002 

0.011 

0.003 

0.000 

#5.1 

0.006 

0.000 

0.004 

0.000 

0.000 

0.008 

0.001 

0.008 

0.003 

0.000 

#6.1 

0.004 

0.001 

0.009 

0.000 

0.000 

0.013 

0.000 

0.001 

0.002 

0.042 

#7.1 

0.003 

0.002 

0.011 

0.002 

0.012 

0.001 

0.022 

0.003 

0.022 

0.004 

#8.1 

0.001 

0.001 

0.050 

0.001 

0.008 

0.006 

0.001 

0.010 

0.001 

0.000 

#9.1 

0.004 

0.000 

0.013 

0.002 

0.000 

0.004 

0.005 

0.050 

0.003 

0.000 

#10.1 

0.001 

0.020 

0.951 

0.005 

0.962 

0.012 

0.006 

0.002 

0.007 

0.049 

#11.1 

0.002 

0.000 

0.011 

0.002 

0.000 

0.002 

0.013 

0.012 

0.016 

0.000 

#12.1 

0.003 

0.001 

0.950 

0.003 

0.013 

0.007 

0.000 

0.003 

0.005 

0.034 

#13.1 

0.004 

0.008 

0.000 

0.027 

0.026 

0.002 

0.017 

0.003 

0.005 

0.026 

#14.1 

0.006 

0.001 

0.003 

0.001 

0.000 

0.005 

0.002 

0.007 

0.008 

0.000 

#15.1 

0.004 

0.000 

0.022 

0.001 

0.001 

0.012 

0.001 

0.010 

0.001 

0.000 

To  perform  this  correlation  analysis  the  observed  data  and  trained  data  were  concatenated 
into  a  single  dataset  with  the  observed  data  variables  given  a  prefix  of  “I”  or  input  to  the  analysis 
and  the  trained  data  variables  given  a  prefix  of  “O”  or  output  to  the  analysis.  This  dataset  is 
loaded  into  SAS  and  a  correlation  table  generated  comparing  each  input  and  output  variable  to 
each  other  using  a  standard  Pearson  Correlation  Coefficient.  The  SAS  run  is  shown  in  Appendix 
H.  From  this  data  a  t-test  is  performed  between  each  input-output  variable  pair  to  determine  if 
there  is  a  significant  correlation  at  the  a  =  0.05  level.  A  standard  decision  rule  hypothesis  is 
established  for  the  test  where  the  “t”  observed  value  is  from  the  SAS  run  and  the  “t”  tabled 
value  from  a  reference  source.  Table  14  shows  the  results  of  the  significance  determinations  for 
all  of  the  DV  Observed-Trained  variable  pairs. 


Table  14  -  Pearson  Correlation  Analysis  of  Observed  To  Trained  JNNS  Outputs 


Input 

Variable 

Label 

Output 

Variable 

Label 

Pearson 

r 

(From 

SAS 

Run) 

P 

value 

Variable  ID 

n 

t 

Observed 

t 

tabled 

Test 

Result, 

Signif¬ 

icant? 

1 

ITP2 

OTP2 

-0.0515 

0.5471 

2  Passage  of 

Lines 

139 

0.6036 

1.97 

No 

2 

ITP3 

OTP3 

0.99832 

<.0001 

3  Movement  to 

139 

201.6707 

1.97 

Yes 
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Input 

Variable 

Label 

Output 

Variable 

Label 

Pearson 

r 

(From 

SAS 

Run) 

P 

value 

Variable  ID 

n 

t 

Observed 

t 

tabled 

Test 

Result, 

Signif¬ 

icant? 

the  line  of 
departure. 

3 

ITP4 

OTP4 

0.9857 

<.0001 

4  Breach  of  main 
obstacle  belt. 

139 

68.4668 

1.97 

Yes 

4 

ITP5 

OTP5 

0.99784 

<.0001 

5  Penetration  of 

defensive 

positions. 

139 

177.7927 

1.97 

Yes 

5 

ITP7 

OTP  7 

0.99782 

<.0001 

7  River  crossing. 

139 

176.9726 

1.97 

Yes 

6 

ITP8 

OTP8 

0.99383 

<.0001 

8  Seizure  of  key 
terrain. 

139 

104.8783 

1.97 

Yes 

7 

ITP10 

OTP  10 

0.99906 

<.0001 

10  Destruction, 
capture,  or 
bypass  of  enemy 
force. 

139 

269.7584 

1.97 

Yes 

8 

ITP11 

OTP  11 

0.99938 

<.0001 

1 1  Fixing  enemy 
in  position. 

139 

332.2366 

1.97 

Yes 

9 

ITP12 

OTP  12 

0.99855 

<.0001 

12 

Synchronization 
with  supporting 
forces. 

139 

217.1144 

1.97 

Yes 

10 

ITP15 

OTP  15 

0.99877 

<.0001 

15  Destruction  of 
first  echelon 
forces. 

139 

235.7718 

1.97 

Yes 

11 

ITP22 

OTP22 

0.99806 

<.0001 

22  Transfer  of 
mission. 

139 

187.6341 

1.97 

Yes 

12 

ITP23 

OTP23 

0.99886 

<.0001 

23.  Planning 

139 

244.9183 

1.97 

Yes 

13 

ITS1 

OTS1 

0.99189 

<.0001 

1  Departure  from 
the  assembly 
area. 

139 

91.3441 

1.97 

Yes 

14 

ITS2 

OTS2 

0.99872 

<.0001 

2  Passage  of 
lines. 

139 

231.1123 

1.97 

Yes 

15 

ITS5 

OTS5 

0.99818 

<.0001 

5  Penetration  of 

defensive 

positions. 

139 

193.7386 

1.97 

Yes 

16 

ITS8 

OTS8 

-0.04592 

0.5914 

8  Seizure  of  Key 
Terrain 

139 

0.5380 

1.97 

No 

17 

ITS  11 

OTS11 

0.99624 

<.0001 

1 1  Fixing  enemy 
in  position. 

139 

134.5935 

1.97 

Yes 

18 

ITS  12 

OTS12 

0.99904 

<.0001 

12 

Synchronization 
with  supporting 
forces. 

139 

266.9296 

1.97 

Yes 

19 

ITS  15 

OTS15 

0.99642 

<.0001 

15  Destruction  of 
first  echelon 
forces. 

139 

137.9543 

1.97 

Yes 

20 

ITS  18 

OTS18 

0.99919 

<.0001 

18  Deception 
activities. 

139 

290.6289 

1.97 

Yes 

21 

ITS  19 

OTS19 

0.99836 

<.0001 

19  Rear 
operations. 

139 

204.1215 

1.97 

Yes 

22 

ITS20 

OTS20 

0.98841 

<.0001 

20  Entry  into  area 

139 

76.2084 

1.97 

Yes 
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r 
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SAS 

Run) 

P 
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Variable  ID 
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t 
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Signif¬ 

icant? 

of  operations. 

23 

ITS22 

OTS22 

0.99121 

<.0001 

22  Transfer  of 
mission. 

139 

87.6946 

1.97 

Yes 

24 

ITS23 

OTS23 

0.99863 

<.0001 

23.  Planning 

139 

223.3770 

1.97 

Yes 

25 

ITA2 

OTA2 

0.98799 

<.0001 

2  Passage  of 
lines. 

139 

74.8401 

1.97 

Yes 

26 

IT  A3 

OTA3 

0.99872 

<.0001 

3  Movement  to 
the  line  of 
departure. 

139 

231.1123 

1.97 

Yes 

27 

ITA7 

OTA7 

0.99755 

<.0001 

7  River  crossing. 

139 

166.9026 

1.97 

Yes 

28 

ITA10 

OTA  10 

0.99663 

<.0001 

10  Destruction, 
capture,  or 
bypass  of  enemy 
force. 

139 

142.2101 

1.97 

Yes 

29 

ITA11 

OTA11 

0.9975 

<.0001 

1 1  Fixing  enemy 
in  position. 

139 

165.2189 

1.97 

Yes 

30 

ITA12 

OTA12 

0.99632 

<.0001 

12 

Synchronization 
with  supporting 
forces. 

139 

136.0568 

1.97 

Yes 

31 

ITA18 

OTA18 

0.99792 

<.0001 

18  Deception 
activities. 

139 

181.1904 

1.97 

Yes 

32 

ITA19 

OTA19 

0.99483 

<.0001 

19  Rear 
operations. 

139 

114.6597 

1.97 

Yes 

33 

ITA21 

OTA21 

0.99931 

<.0001 

21  Peacekeeping 
operations. 

139 

314.9169 

1.97 

Yes 

34 

ITA23 

OTA23 

0.99915 

<.0001 

23.  Planning 

139 

283.6997 

1.97 

Yes 

35 

IM1 

OM1 

0.99813 

<.0001 

1-  Pre  Operations 
Planning 

139 

191.1238 

1.97 

Yes 

36 

IM2 

OM2 

0.99073 

<.0001 

2-  Movement  to 
Contact 

139 

85.3630 

1.97 

Yes 

37 

IM3 

OM3 

0.99664 

<.0001 

3-  Attack 

139 

142.4227 

1.97 

Yes 

38 

IM4 

OM4 

0.96609 

<.0001 

4-  Defense 

139 

43.7938 

1.97 

Yes 

39 

IM5 

OM5 

0.99683 

<.0001 

5-  River  Crossing 

139 

146.6497 

1.97 

Yes 

Table  14  shows  a  high  correlation  for  all  of  the  DV  input-output  pairs  except  ITP2-OTP2 
Passage  of  Lines,  and  ITS8-OTS8,  Seizure  of  Key  Terrain.  An  examination  of  the  original 
observation  data  for  these  two  variable  pairs  provides  the  explanation  of  their  non-correlation. 
During  the  original  observation  of  the  TOC  exercises  these  two  variables  were  only  observed  to 
occur  two  or  three  times  with  the  result  that  there  was  insufficient  data  for  the  JNNS  training 
algorithm  to  achieve  significance. 

This  correlation  analysis  provides  a  high  degree  confidence  in  the  accuracy  of  the  state  of 
training  in  the  JNNS  simulation  as  the  analysis  proceeds  to  the  pruning  stage. 
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3.2.2.  JNNS  Network  Pruning. 


Pruning,  in  neural  network  terminology,  is  the  process  of  identifying  those  input  nodes 
that  are  predicted  to  be  less  significant  than  others  in  contributing  to  the  performance  of  the 
network  in  the  replication  of  the  trained  network  to  the  observed  data.  The  training  process 
previously  performed  has  tuned  the  NNS  simulation  model  to  represent  the  interactions  between 
the  input  nodes  and  the  output  nodes  to  a  certain  acceptance  level.  Pruning  is  where  nodes  are 
eliminated  according  to  a  performance  threshold.  Through  multiple  pruning  iterations  nodes  are 
identified  that  are  potential  candidates  to  be  declared  less  important  than  other  nodes.  Figure  34 
shows  an  example  of  the  JNNS  TOC  network  after  a  successful  pruning  operation. 
Superimposed  on  the  chart  are  the  threshold  parameters  set  into  JNNS  for  the  run.  Those  nodes 
that  have  been  eliminated  are  shown  with  their  links  removed. 
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While  this  display  is  visually  informative,  the  exact  state  of  the  node  relationships  is  indicated  by 
text  results  files.  During  execution  INNS  saves  its  network  design  into  ASCII  text  files  that 
includes  a  bias  parameter  containing  the  calculated  link  coefficient  detennined  during  training. 
During  the  pruning  stage  if  the  node  is  eliminated  then  this  bias  coefficient  is  set  to  zero.  Table 
15  shows  the  bias  information  for  this  network  before  and  after  pruning. 
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Table  15  -  JNNS  TOC  Simulation  Bias  After  Pruning 


Node  no. 

Unit  Name 

Trained  bias 

Pruned  bias 

1 

Dayl 

-0.98737 

-0.9975 

2 

Day2 

-0.92248 

0.12717 

3 

Day3 

0.41398 

-0.61339 

4 

Day  4 

-0.56047 

0.61748 

5 

Day5 

-0.62572 

0.17002 

6 

Day  6 

-0.81347 

-0.04025 

7 

Day7 

-0.05356 

-0.29942 

8 

Day8 

0.93939 

0.79192 

9 

Day  9 

-0.44438 

0.64568 

10 

DaylO 

-0.31748 

0.49321 

11 

Dayl  1 

0.09232 

0 

12 

Day  12 

-0.66192 

0 

13 

Dayl  3 

0.63536 

0 

14 

Dayl  4 

-0.73986 

0 

15 

Dayl  5 

-0.02396 

0 

16 

Dayl  6 

0.63854 

0 

17 

Dayl  7 

-0.52257 

0 

18 

Dayl  8 

-0.28916 

0 

19 

Dayl  9 

0.07138 

0 

20 

Day20 

0.87304 

0 

21 

Hour  1 

0.62236 

0 

22 

Hour2 

0.50078 

0 

23 

Hour3 

-0.28043 

0 

24 

Hour4 

-0.19962 

0 

25 

Hour5 

0.44682 

0 

26 

Hour6 

0.53917 

0 

27 

Hour7 

-0.92743 

0.06333 

28 

Hour8 

-0.67498 

0.14237 

29 

Hour9 

-0.84307 

0.20353 

30 

Hour  10 

0.45128 

0.21433 

31 

Hourl  1 

-0.47478 

-0.66753 

32 

Hour  12 

0.45109 

0.32609 

33 

Hour  13 

-0.82531 

-0.09842 

34 

Hour  14 

-0.33256 

-0.29575 

35 

Hour  15 

-0.30235 

-0.88592 

36 

Hour  16 

-0.41539 

0.21537 

37 

Hour  17 

-0.51799 

0.56664 

38 

Hour  18 

-0.35899 

0.60521 

39 

Hour  19 

-0.96869 

0 

40 

Hour20 

-0.00424 

0 

41 

Hour2 1 

0.48454 

0 

42 

Hour22 

-0.49205 

0 

43 

Hour23 

0.06101 

0 

44 

Hour24 

-0.74664 

0 

45 

AnalogTOC 

0.95685 

0 

46 

DigitalTOC 

-0.49998 

0 

47 

BnTOC 

-0.57561 

0 

48 

BdeTOC 

0.4453 

0 
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Node  no. 

Unit  Name 

Trained  bias 

Pruned  bias 

49 

Test  Ex 

-0.36729 

0.72448 

50 

BCTP  Ex 

0.4246 

-0.5808 

51 

NTC  Ex 

0.67315 

0 

52 

Tng  Ex 

-0.00436 

0.68731 

53 

ExNo  1 

-0.99768 

0 

54 

ExNo  2 

0.52312 

0 

55 

ExNo  3 

0.94678 

0 

56 

ExNo  4 

-0.10709 

0 

57 

ExNo  5 

-0.12125 

0 

58 

ExNo  6 

0.02145 

0 

59 

ExNo  7 

0.5728 

0 

60 

ExNo  8 

-0.85998 

0 

61 

ExNo  9 

0.29673 

0 

62 

ExNo  10 

0.68365 

0 

63 

ExNo  11 

-0.3455 

0 

64 

ExNo  12 

0.24058 

0 

65 

ExNo  13 

-0.69433 

0 

66 

ExNo  14 

0.68236 

0 

67 

ExNo  15 

0.71026 

0 

68 

ExNo  16 

-0.46886 

0 

69 

ExNo  17 

-0.7796 

0 

70 

ExNo  18 

0.09799 

0 

71 

ExNo  19 

-0.57469 

0 

72 

ExNo  20 

-0.82434 

0 

73 

SimNone 

0.89038 

0 

74 

SimJANUS 

-0.64043 

0 

75 

SimJAN  STRM 

0.58007 

0 

76 

SimCBS 

-0.36912 

0 

77 

SimCCTT 

-0.44829 

0 

78 

SimSTORM 

-0.0607 

0 

79 

BOS-Intell 

-0.78387 

0.48888 

80 

BOS-Intel2 

0.39775 

-0.78344 

81 

BOSIntel3 

-0.17637 

0.1981 

82 

BOSInteM 

-0.3867 

-0.22953 

83 

BOS-MAN1 

0.10361 

0.47002 

84 

BOS-MAN2 

-0.1298 

0.21793 

85 

BOS-FS1 

-0.58824 

0 

86 

BOS-FS2 

-0.39818 

-0.27732 

87 

BOS-FS3 

-0.42888 

0 

88 

BOS-FS4 

0.09452 

0 

89 

BOS-FS5 

0.0809 

0 

90 

BOS-FS6 

-0.09977 

0 

91 

BOS-FS7 

0.7622 

0.03421 

92 

BOS-FS8 

-0.74938 

0 

93 

BOS-FS9 

-0.5399 

0.5031 

94 

BOS-MAS1 

0.1146 

-0.30888 

95 

BOS-MAS2 

0.37938 

-0.66204 

96 

BOS-MAS3 

-0.33091 

0 

97 

BOS-MAS4 

0.24223 

-0.01621 

98 

BOS-MAS5 

0.11295 

-0.87292 

99 

BOS-MAS6 

-0.02036 

0.39952 

100 

BOS-MAS7 

0.19572 

0.00961 
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Node  no. 

Unit  Name 

Trained  bias 

Pruned  bias 

101 

BOS-ADA1 

0.34001 

-0.70501 

102 

BOS-ADA2 

-0.19413 

0.89917 

103 

BOS-C2-1 

-0.11728 

-0.71685 

104 

BOS-C2-2 

0.71606 

0.81024 

105 

BOS-C2-3 

0.56273 

0.38578 

106 

BOSR-Intel 

0.36876 

-0.3939 

107 

BOSR-MAN 

0.75542 

0 

108 

BOSR-FS 

-0.34721 

0 

109 

BOSR-MAS 

-0.62645 

0.93323 

110 

BOSR-ADA 

0.21708 

0 

111 

BOSR-CSS 

0.93377 

0 

112 

BOSR-C2 

0.24827 

0 

113 

COM-AFATDS 

-0.70714 

0.64336 

114 

COM-AMPS 

0.0491 

0 

115 

COM-ASAS 

0.54698 

-0.6173 

116 

COM-CTIS 

0.4445 

0 

117 

COM-CSSCS 

0.50725 

0 

118 

COM-FAADC2 

0.2667 

0 

119 

COM-FBCB2 

0.50334 

-0.68889 

120 

COM-IMETS 

0.17344 

0 

121 

COM-MCS 

0.24894 

0.46403 

122 

COM-AMDWS 

-0.9118 

-0.18882 

123 

COM-JSTARS 

-0.59697 

0 

124 

COM-EPLARS 

0.87005 

0 

125 

INTEL-TUAV 

-0.49297 

0.36448 

126 

INTEL-JSTARS 

0.6646 

0 

127 

INTEL-GSR 

0.43852 

0 

128 

MISC-Timing 

-0.97967 

-0.04941 

129 

MISC-Tempo 

-0.36699 

-0.75396 

130 

MISC-ReconOps 

-0.48753 

-0.26438 

131 

MISC-InfoOps 

-0.65392 

0.66936 

132 

MISC-Tactics 

0.55724 

0 

133 

MISC-TOC  Activity 

0.40379 

0.03403 

134 

MOPP  Level 

0.57585 

0.32597 

135 

Staff_FIiiddle 

-0.52947 

-0.14756 

136 

Pers-CO 

-0.0839 

-0.79064 

137 

Pers-XO 

0.10361 

0.89868 

138 

Pers-BC 

-0.77074 

0.84277 

139 

Pers-S3 

-0.99457 

0.09909 

140 

Pers-S3  NCO 

0.56377 

-0.30802 

141 

Pers-S3  RTO 

-0.8114 

-0.05655 

142 

PERS-S2 

0.53502 

-0.25004 

143 

PERS-FSO 

0.47233 

0.69396 

144 

PERS-ENGR 

0.81188 

-0.36625 

145 

PERS-ALO 

0.09568 

-0.0878 

146 

PERS-S3-SGM 

0.06632 

-0.45622 

147 

SKY-CLEAR 

-0.50023 

0.96594 

148 

SKY-OVERCAST 

-0.27134 

-0.4044 

149 

SKY-RAIN 

0.75817 

0 

150 

WIND-NONE 

-0.38993 

0.13456 

151 

WIND-LOW 

0.96655 

-0.60802 

152 

WIND-MODERATE 

0.96008 

0.52263 
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Node  no. 

Unit  Name 

Trained  bias 

Pruned  bias 

153 

WIND-HIGH 

0.39128 

0 

154 

Light-BG 

0.0604 

0 

155 

Light-DISPLAY 

-0.42998 

0 

156 

NOISE 

-0.72582 

0 

157 

TEMP-DB 

-0.89026 

0 

158 

TEMP-WB 

0.81652 

0 

159 

Humidity 

0.88336 

0.14518 

Table  15  provides  the  set  of  those  IV  to  be  carried  forward  in  the  analysis  as  those  IV  with  a 
pruned  bias  greater  than  zero.  Some  of  the  eliminated  IV  nodes  are  intuitively  obvious.  For 
example,  the  IV  for  exercise  day  was  eliminated  after  day  1 1 .  This  reflects  the  fact  that  no 
exercise  had  longer  than  a  10  day  recording  period.  Circadian  rhythm  information  variables 
eliminated  all  hours  of  the  day  except  for  hours  7:00  a.m.  through  6:00  p.m.  Because  only  one 
person  was  collecting  data  there  was  no  data  collected  at  hours  other  than  these.  However,  many 
other  perfonnance  parameters  that  were  not  immediately  obvious  were  eliminated  by  the  JNNS 
model.  Many  of  the  artillery  battlefield  operating  systems  (BOS-FS)  IV  were  shown  to  be  less 
significant.  Many  of  the  communications  systems  (COM-xxx)  were  identified  as  less  significant 
than  others.  This  was  partly  due  to  the  fact  that  observation  abilities  limited  the  scope  of  data 
that  could  be  collected,  but  also  it  was  due  to  the  observations  that  were  observed  of  several  of 
the  more  significant  systems.  All  of  the  personnel  IV  remained  after  pruning.  Most  of  the 
environmental  variables,  with  the  exception  of  the  wind  and  humidity  factors,  were  removed  by 
the  pruning  calculations. 

3.2.3.  Development  of  Mathematical  Models  of  DV  Performance  From  JNNS. 

As  the  JNNS  simulation  is  not  capable  of  generating  descriptive  expressions  of  how  the 
DV  respond  to  the  various  IV,  an  alternative  procedure  was  established  to  develop  these 
mathematical  relationships.  The  first  step  was  to  identify  those  DV  for  which  sufficient  data  had 
been  collected  to  allow  an  attempt  to  develop  a  model  of  their  performance.  A  subjective 
judgment  was  made  that  there  must  be  at  least  10  observations  that  were  made  against  a  DV  to 
allow  sufficient  data  for  it  to  be  modeled.  This  examination  of  the  data  was  performed  with  a 
SAS  correlation  run  of  each  DV  to  each  of  the  other  DV  that  generated  simple  descriptive 
statistics  for  all  the  DV.  One  of  the  parameters  that  this  run  produced  was  the  sum  of  the  data 
values  for  each  DV.  For  the  binary  DV  this  sum  provided  a  count  of  the  number  of  observations 
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in  the  dataset  for  that  DV.  For  the  standardized  (0,1)  DV,  this  sum  provided  a  value  that 
approached  the  number  of  observations  for  that  DV  in  the  dataset.  Table  16  shows  the  list  of  the 
17  DV  that  were  selected  for  modeling  by  selecting  those  DV  with  a  sum  of  10  or  greater. 
Appendix  I  contains  the  output  from  the  SAS  run  that  was  used  to  determine  this  list  of  DV. 


Table  16  -  DV  Selected  For  Model  Generation 


Model 

Category 

Name 

Label 

1 

Primary  Task 

10  Destruction,  capture,  or  bypass  of  enemy  force. 

TP  10 

2 

Primary  Task 

1 1  Fixing  enemy  in  position. 

TP1 1 

3 

Primary  Task 

12  Synchronization  with  supporting  forces. 

TP12 

4 

Primary  Task 

15  Destruction  of  first  echelon  forces. 

TP  15 

5 

Primary  Task 

21  Peacekeeping  operations. 

TP21 

6 

Primary  Task 

23.  Planning 

TP23 

7 

Secondary  Task 

12  Synchronization  with  supporting  forces. 

TS12 

8 

Secondary  Task 

18  Deception  activities. 

TS18 

9 

Secondary  Task 

19  Rear  operations. 

TS19 

10 

Secondary  Task 

23.  Planning 

TS23 

11 

Tertiary  Task 

3  Movement  to  the  line  of  departure. 

TA3 

12 

Tertiary  Task 

21  Peacekeeping  operations. 

TA21 

13 

Tertiary  Task 

23.  Planning 

TA23 

14 

Mission 

1-  Pre  Operations  Planning 

Mission  1 

15 

Mission 

2-  Movement  to  Contact 

Mission2 

16 

Mission 

3-  Attack 

Mission3 

17 

Mission 

5-  River  Crossing 

Mission5 

For  each  DV  to  be  modeled,  the  next  step  was  to  identify  those  IV  that  are  predicted  to 
significantly  contribute  to  a  model  of  the  DV.  This  was  performed  through  simple  linear 
regression  analysis.  Using  SAS,  an  initial  model  of  each  of  the  DV  was  constructed  using  IV 
remaining  from  the  JNNS  pruning  stage.  Those  IV  that  are  predicted  to  have  significance  for 
model  generation  are  shown  in  Table  17.  In  addition,  administrative  variables  such  as  day  of  the 
exercise,  hour  of  the  day,  type  of  TOC,  exercise  number,  and  type  of  simulation  were  also 
eliminated  as  they  were  recorded  in  the  observation  data  for  possible  future  analysis. 
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Table  17  -  IV  Selected  For  Inclusion  in  DV  Models  Based  On  JNNS  Pruning 


Node  no. 

Variable  Name 

Trained  bias 

Pruned  bias 

79 

BOS-Intell 

-0.78387 

0.48888 

80 

BOS-Intel2 

0.39775 

-0.78344 

81 

BOSIntel3 

-0.17637 

0.1981 

82 

BOSInteM 

-0.3867 

-0.22953 

83 

BOS-MAN1 

0.10361 

0.47002 

84 

BOS-MAN2 

-0.1298 

0.21793 

86 

BOS-FS2 

-0.39818 

-0.27732 

91 

BOS-FS7 

0.7622 

0.03421 

93 

BOS-FS9 

-0.5399 

0.5031 

94 

BOS-MAS1 

0.1146 

-0.30888 

95 

BOS-MAS2 

0.37938 

-0.66204 

97 

BOS-MAS4 

0.24223 

-0.01621 

98 

BOS-MAS5 

0.11295 

-0.87292 

99 

BOS-MAS6 

-0.02036 

0.39952 

100 

BOS-MAS7 

0.19572 

0.00961 

101 

BOS-ADA1 

0.34001 

-0.70501 

102 

BOS-ADA2 

-0.19413 

0.89917 

103 

BOS-C2-1 

-0.11728 

-0.71685 

104 

BOS-C2-2 

0.71606 

0.81024 

105 

BOS-C2-3 

0.56273 

0.38578 

106 

BOSR-Intel 

0.36876 

-0.3939 

109 

BOSR-MAS 

-0.62645 

0.93323 

113 

COM-AFATDS 

-0.70714 

0.64336 

115 

COM-ASAS 

0.54698 

-0.6173 

119 

COM-FBCB2 

0.50334 

-0.68889 

121 

COM-MCS 

0.24894 

0.46403 

122 

COM-AMDWS 

-0.9118 

-0.18882 

125 

INTEL-TUAV 

-0.49297 

0.36448 

128 

MISC-Timing 

-0.97967 

-0.04941 

129 

MISC-Tempo 

-0.36699 

-0.75396 

130 

MISC-ReconOps 

-0.48753 

-0.26438 

131 

MISC-InfoOps 

-0.65392 

0.66936 

133 

MISC-TOC  Activity 

0.40379 

0.03403 

134 

MOPP  Level 

0.57585 

0.32597 

135 

Staff  Ftuddle 

-0.52947 

-0.14756 

136 

Pers-CO 

-0.0839 

-0.79064 

137 

Pers-XO 

0.10361 

0.89868 

138 

Pers-BC 

-0.77074 

0.84277 

139 

Pers-S3 

-0.99457 

0.09909 

140 

Pers-S3  NCO 

0.56377 

-0.30802 

141 

Pers-S3  RTO 

-0.8114 

-0.05655 

142 

PERS-S2 

0.53502 

-0.25004 

143 

PERS-FSO 

0.47233 

0.69396 

144 

PERS-ENGR 

0.81188 

-0.36625 

145 

PERS-ALO 

0.09568 

-0.0878 

146 

PERS-S3-SGM 

0.06632 

-0.45622 

147 

SKY-CLEAR 

-0.50023 

0.96594 

148 

SKY-OVERCAST 

-0.27134 

-0.4044 

174 


Node  no. 

Variable  Name 

Trained  bias 

Pruned  bias 

150 

WIND-NONE 

-0.38993 

0.13456 

151 

WIND-LOW 

0.96655 

-0.60802 

152 

WIND-MODERATE 

0.96008 

0.52263 

159 

Humidity 

0.88336 

0.14518 

An  initial  SAS  model  was  constructed  for  each  of  the  DV  containing  each  of  the  IV  and  a 
simple  linear  regression  was  executed.  The  results  of  the  first  regression  run  are  shown  in  Table 
18  using  Primary  Task  10  (TP  10-  Destruction,  capture,  or  bypass  of  enemy  force)  as  an  example. 
The  results  for  all  17  DV  is  at  Appendix  J. 
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Table  18  -  First  Iteration  To  Select  IV  For  Inclusion  in  TP10  DV  Model  Based  On  JNNS 
Pruning 


Regression  Analysis  for  Exercises  1,2,3 
The  REG  Procedure 
Model:  MODEL1 
Dependent  Variable:  TP10 
Analysis  of  Variance 


Sum  of 

Mean 

Source 

DF 

Squares 

Square 

F  Value 

Pr  > 

Model 

46 

12 . 02080 

0.26132 

17 . 67 

<.000 

Error 

92 

1.36049 

0 . 01479 

Corrected  Total 

138 

13.38129 

Root  MSE 

0 . 12161 

R-Square 

0.8983 

Dependent 

Mean 

0 . 10791 

Adj  R-Sq 

0 . 8475 

Coeff  Var 

112 . 68797 

Parameter  Estimates 

Parameter 

Standard 

Variable 

DF 

Estimate 

Error 

t  Value 

Pr  >  | t | 

Intercept 

1 

0 . 14508 

0.40682 

0.36 

0 . 7222 

BOSIntell 

1 

0 . 54077 

0 . 15107 

3.58 

0.0006 

BOSIntel2 

1 

0 . 64916 

0.13633 

4 .76 

<.0001 

BOSIntel3 

1 

0.62009 

0.13999 

4 . 43 

<.0001 

BOSIntel4 

1 

0 . 72621 

0 . 17174 

4.23 

<.0001 

BOSManl 

1 

-0 . 14944 

0.13684 

-1.09 

0.2777 

BOSMan2 

1 

0.03083 

0 . 12721 

0.24 

0.8090 

BOSFS2 

1 

0 . 11876 

0.06996 

1 . 70 

0.0930 

BOSFS7 

1 

-0 . 10047 

0.08353 

-1.20 

0.2321 

BOSFS9 

1 

-0.02630 

0 . 07945 

-0.33 

0 . 7414 

BOSMAS1 

1 

0 . 04637 

0.07388 

0.63 

0.5319 

BOSMAS2 

1 

0.22815 

0.08037 

2 . 84 

0.0056 

BOSMAS4 

1 

0 . 10387 

0 .11042 

0.94 

0.3493 

BOSMAS5 

1 

-0 . 01027 

0 . 04040 

-0.25 

0.8000 

BOSMAS6 

1 

0.62889 

0.20857 

3.02 

0.0033 

BOSMAS7 

1 

0.22498 

0 . 12430 

1.81 

0.0736 

BOSADA1 

1 

-1.24598 

0.39498 

-3.15 

0 . 0022 

BOSADA2 

1 

-1 . 13750 

0 . 37516 

-3.03 

0.0032 

BOSCC1 

1 

-0 . 39701 

0.13599 

-2 . 92 

0 . 0044 

BOSCC2 

1 

-0.50633 

0 . 13754 

-3.68 

0.0004 

BOSCC3 

1 

-0 . 42400 

0 .16674 

-2 . 54 

0 . 0127 

BRIntel 

1 

-0 .01445 

0 .11059 

-0.13 

0.8964 

BRMAS 

1 

-0.33434 

0.23039 

-1 . 45 

0 . 1501 

COMAFATDS 

1 

-0.30881 

0 . 15894 

-1 . 94 

0 . 0551 

COMASAS 

1 

-0.19206 

0 . 18605 

-1.03 

0.3046 

COMMCS 

1 

-0 .06422 

0 .16817 

-0.38 

0 . 7034 

COMAMDWS 

1 

0.09252 

0 .16508 

0.56 

0 . 5765 

INTELTUAV 

1 

-0 .06244 

0.08539 

-0.73 

0 .4665 

MISCBattleTiming 

1 

0 . 40262 

0.36346 

1 . 11 

0.2709 

MISCBattleTempo 

1 

-0 . 33771 

0.27539 

-1.23 

0.2232 

MISCReconOps 

1 

0.04683 

0.06986 

0.67 

0 . 5043 

MISCInfoOps 

1 

-0 . 05997 

0 .09474 

-0.63 

0.5283 

MISCTOCAct ivity 

1 

-0.00732 

0.08631 

-0.08 

0.9326 

MOPPLevel 

1 

2 . 54402 

0 .89515 

2 . 84 

0 . 0055 

Staf fHud 

1 

0 . 01922 

0 . 05865 

0.33 

0 . 7438 

PERSCMDR 

1 

-0 . 08179 

0 . 04877 

-1.68 

0.0969 

PERSXO 

1 

0 . 04952 

0 . 04542 

1.09 

0.2785 

PERSBC 

1 

0.03228 

0 .06556 

0.49 

0.6237 

PERSS3 

1 

0.09948 

0 .06354 

1 . 57 

0.1209 

PERSS3NCO 

1 

-0.02453 

0.06618 

-0.37 

0 . 7117 

PERSS3RTO 

1 

0 . 03501 

0.06963 

0.50 

0.6163 

PERSS2 

1 

0.00925 

0 . 04374 

0.21 

0.8330 

PERSFSO 

1 

-0.03583 

0 . 05340 

-0.67 

0.5039 

PERSENGR 

1 

0.05230 

0.06225 

0.84 

0 .4029 

PERSALO 

1 

-0.06108 

0 . 04341 

-1 . 41 

0 .1627 

PERSS3SGM 

1 

-0.00878 

0 . 04948 

-0.18 

0.8596 

Humidity 

1 

-0 .01476 

0 . 12172 

-0 . 12 

0.9037 
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The  results  of  the  second  iterative  regression  run  are  shown  in  Table  19  using  Primary 
Task  10  (TP  10-  Destruction,  capture,  or  bypass  of  enemy  force)  as  an  example.  The  results  for 
all  17  DV  is  at  Appendix  K. 

Table  19  -  Second  Iteration  To  Select  IV  For  Inclusion  in  TP10  DV  Model  Based  On  JNNS 
Pruning 


Second  level  Regression  Analysis  for  Exercises  1,2,3  51 

12:04  Wednesday,  December  18,  2002 

The  REG  Procedure 
Model:  M0DEL1 
Dependent  Variable:  TP10 
Analysis  of  Variance 


Sum  of 

Mean 

Source 

DF 

Squares 

Square 

F  Value 

Pr  >  1 

Model 

10 

7.97293 

0.79729 

18.87 

<.000 

Error 

128 

5.40837 

0.04225 

Corrected  Total 

138 

13.38129 

Root  MSE 

0.20556 

R-Square 

0.5958 

Dependent 

Mean 

0.10791 

Adj  R-Sq 

0.5643 

Coeff  Var 

190.48100 

Parameter  Estimates 

Parameter 

Standard 

Variable 

DF 

Estimate 

Error 

t  Value 

Pr  >  1 1 1 

Intercept 

1 

0.44886 

0.17174 

2.61 

0.0100 

B0SIntel3 

1 

0.00466 

0.05049 

0.09 

0.9266 

B0SIntel4 

1 

0.27463 

0.08769 

3.13 

0.0022 

B0SMAS2 

1 

0.25440 

0.06685 

3.81 

0.0002 

B0SMAS6 

1 

-0.21173 

0.14555 

-1.45 

0.1482 

B0SADA1 

1 

0.22378 

0.16293 

1  .37 

0.1720 

B0SADA2 

1 

-0.01263 

0.12326 

-0.10 

0.9186 

B0SCC1 

1 

-0.22450 

0.14833 

-1.51 

0.1326 

B0SCC2 

1 

-0.53501 

0.16576 

-3.23 

0.0016 

B0SCC3 

1 

-0.34403 

0.17206 

-2.00 

0.0477 

MOPPLevel 

1 

-0.68254 

0.25038 

-2.73 

0.0073 

The  results  of  the  third  iterative  regression  run  are  shown  in  Table  20  using  Primary  Task 
10  (TP  10-  Destruction,  capture,  or  bypass  of  enemy  force)  as  an  example.  The  results  for  all  17 
DV  is  at  Appendix  L. 
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Table  20  -  Third  Iteration  To  Select  IV  For  Inclusion  in  TP10  DV  Model  Based  On  JNNS 
Pruning 


Third  level  Regression  Analysis  for  Exercises  1,2,3  1 

12:04  Wednesday,  December  18,  2002 

The  REG  Procedure 
Model:  MODEL1 
Dependent  Variable:  TP10 
Analysis  of  Variance 


Sum  of 

Mean 

Source 

DF 

Squares 

Square 

F  Value 

Pr  >  F 

Model 

5 

7.51355 

1 .50271 

34.06 

<.0001 

Error 

133 

5.86775 

0.04412 

Corrected  Total 

138 

13.38129 

Root  MSE 

0.21004 

R-Square 

0.5615 

Dependent 

Mean 

0.10791 

Adj  R-Sq 

0.5450 

Coeff  Var 

194.64058 

Parameter  Estimates 

Parameter 

Standard 

Variable 

DF 

Estimate 

Error 

t  Value 

Pr  >  1 1 1 

Intercept 

1 

0.06734 

0.05007 

1  .35 

0.1809 

B0SIntel4 

1 

0.36773 

0.04806 

7.65 

<.0001 

B0SMAS2 

1 

0.30461 

0.05644 

5.40 

<.0001 

B0SCC2 

1 

-0.16734 

0.05570 

-3.00 

0.0032 

B0SCC3 

1 

0.04340 

0.05602 

0.77 

0.4399 

MOPPLevel 

1 

-0.43653 

0.12104 

-3.61 

0.0004 

The  results  of  the  fourth  iterative  regression  run  are  shown  in  Table  21  using  Primary 
Task  10  (TP  10-  Destruction,  capture,  or  bypass  of  enemy  force)  as  an  example.  The  results  for 
all  17  DV  is  at  Appendix  M.  The  results  from  this  regression  run  show  that  all  the  remaining  IV 
for  each  of  the  DV  models  have  achieved  significance  at  the  a  =  .05  level.  Thus,  the  parameter 
estimates,  or  Beta  Weights,  along  with  the  intercept  in  the  SAS  output  for  each  DV  form  the 
linear  regression  equation,  or  mathematical  model,  that  describe  the  input  from  each  significant 
IV  to  each  of  the  DV  models. 
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Table  21  -  Fourth  Iteration  To  Select  IV  For  Inclusion  in  TP10  DV  Model  Based  On  JNNS 
Pruning 


Fourth  level  Regression  Analysis  for  Exercises  1,2,3  1 

12:04  Wednesday,  December  18,  2002 

The  REG  Procedure 
Model:  M0DEL1 
Dependent  Variable:  TP10 
Analysis  of  Variance 


Sum  of 

Mean 

Source 

DF 

Squares 

Square 

F  Value 

Pr  >  F 

Model 

4 

7.48707 

1 .87177 

42.55 

<.0001 

Error 

134 

5.89423 

0.04399 

Corrected  Total 

138 

13.38129 

Root  MSE 

0.20973 

R-Square 

0.5595 

Dependent 

Mean 

0.10791 

Adj  R-Sq 

0 . 5464 

Coeff  Var 

194.34998 

Parameter  Estimates 

Parameter 

Standard 

Variable 

DF 

Estimate 

Error 

t  Value 

Pr  >  1 1 1 

Intercept 

1 

0.09781 

0.03094 

3.16 

0.0019 

B0SIntel4 

1 

0.37701 

0.04647 

8.11 

<.0001 

B0SMAS2 

1 

0.31685 

0.05410 

5.86 

<.0001 

B0SCC2 

1 

-0.19909 

0.03768 

-5.28 

<.0001 

MOPPLevel 

1 

-0.46896 

0.11340 

-4.14 

<.0001 

From  these  results  the  linear  regression  equation  for  the  TP  10  variable  is  represented  in 
Table  22.  The  similar  derivation  of  linear  regression  expressions  for  all  the  DV  models  is  at 
Appendix  N. 

Table  22  -  Linear  Regression  Expression  For  DV  TP  10 


#  1  -  Primary  Task  10.  Destruction,  capture,  or  bypass  ol 

'  enemy  force. 

Variable 

Intercept 

BOSInteM 

BOSMAS2 

BOSCC2 

MOPPLevel 

TP  10 

Beta  Weight 

0.09781 

0.37701 

0.31685 

-0.19909 

-0.46896 

3.2.4.  Summary  of  Results. 

The  use  of  neural  networks,  as  implemented  by  JNNS,  along  with  post  processing  of  its 
output  with  traditional  linear  regression  modeling,  provides  a  straightforward  approach  to 
analyzing  the  data.  The  intent  is  to,  first,  identify  those  IV  that  are  predicted  to  significantly 
affect  the  outcomes  of  the  system.  JNNS  provides  this  ability  through  its  pruning  capability  after 
successful  training  of  the  network.  In  fact,  JNNS  is  being  used  here  as  a  factor  screening  method 
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to  identify  those  relationships  contained  in  the  dataset  that  potentially  affect  the  perfonnance  of 
the  system.  After  those  IV  that  are  predicted  to  be  less  important  were  rejected  during  the 
pruning  process,  the  dataset  became  more  manageable  with  traditional  modeling  approaches  such 
as  stepwise  linear  regression.  The  results  of  the  regression  analysis  provided  the  algorithmic 
models  for  use  in  the  remainder  of  the  COMPASS  framework  to  provide  control  inputs  for  the 
CoHOST  discrete  event  simulation  of  the  C2S. 
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4.  Phase  II  -  Development  of  Algorithmic  Control  Interfaces  for  TOC  Discrete  Event 

Simulations. 

Phase  II  of  this  research  consisted  of  applying  the  mathematical  control  models 
developed  in  Phase  I  into  a  suitable  simulation  of  the  perfonnance  of  the  overall  system.  This 
assumes  that  a  simulation  of  this  type  already  exists  which  is  the  case  here.  This  discrete  event 
simulation  is  the  CoHOST  model  previously  developed  by  the  U.S.  Army  (Middlebrooks,  2001; 
Middlebrooks  et  al.,  1999a;  Middlebrooks  et  al.,  1999b).  CoHOST  provides  simulated  human 
test  subjects  who  act  as  avatars  to  evaluate  the  effects  of  the  different  IV  through  experimental 
evaluations  of  the  data.  Appendix  O  provides  an  overall  description  of  the  CoHOST  simulation 
for  readers  unfamiliar  with  it  along  with  a  brief  description  of  the  results  from  the  project  that 
developed  it. 

The  interface  of  the  mission  based  mathematical  performance  models  from  Phase  I  to  the 
operator  task  and  workload  performance  structure  of  the  CoHOST  model  is  a  critical  component 
of  the  overall  COMPASS  framework.  In  the  following  sections  procedures  are  developed  that 
enable  CoHOST  to  respond  to  mission  based  stressors  so  that  the  use  of  the  simulation  can  be 
extended  to  include  predictions  of  the  effectiveness  of  the  different  subsystems  and  associated 
performance  measures  that  exist  within  the  C2S. 

4.1.  Method. 

The  methodology  for  this  research  is  the  implementation  of  the  mathematical  models 
developed  by  the  JNNS  simulation  into  a  form  usable  by  the  CoHOST  simulation.  The  interface 
points  were  first  developed  in  the  last  part  of  Phase  I.  These  interfaces  take  observational  data  of 
mission  based  overall  system  level  performance  of  the  TOC  and  correlates  it  to  task  and 
workload  perfonnance  characteristics  for  the  human  operators  in  the  TOC.  CoHOST  simulation 
runs  are  made  according  to  the  interface  parameters  and  the  results  of  the  runs  are  analyzed  to 
determine  the  predicted  effects  from  changes  in  mission  characteristics. 

4.1.1.  Development  of  JNNS  to  CoHOST  Interface. 

The  process  to  translate  the  results  of  the  JNNS  (Fischer  et  al.,  2001)  into  performance 
parameters  that  detennine  how  a  discrete  event  simulation  of  the  C2  system  performs  is  critical 
to  the  establishment  of  the  COMPASS  paradigm.  While  the  C2  system  field  observations  that 
are  evaluated  by  the  neural  network  are  characterized  as  mission  based  performance  parameters 
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of  the  overall  system,  the  CoHOST  discrete  event  simulation  models  the  C2  system  in  terms  of 
human  performance,  workload,  and  utilization  of  the  human  operators  in  the  system.  The  overall 
intent  of  the  COMPASS  approach  is  to  systematically  evaluate  the  C2  system  so  that  human 
performance  can  be  quantitatively  described  within  that  system.  Once  this  goal  is  achieved  the 
COMPASS  approach  can  be  used  to  evaluate  modifications  of  that  system  in  terms  of  the  human 
ability  to  perform  where  the  proposed  changes  to  the  system  are  compared  to  the  baseline  as 
characterized  by  the  current  existing  system. 

The  correlation  of  the  mission  based  JNNS  data  structures  to  the  human  operator  task  and 
workload  based  CoHOST  data  structures  requires  a  procedural  approach  where  JNNS  mission 
performance  is  cross  tabulated  with  CoHOST  performance  shaping  factors  (PSF)  that  modify  the 
amount  of  time  required  to  perfonn  a  task.  The  resulting  operator  workload  is  determinant  upon 
the  instantaneous  workload  as  determined  by  a  knowledge,  skills,  and  abilities  assessment  using 
the  operator  performance  categories  of  gross  motor,  psychomotor,  auditory,  visual, 
communications,  conceptual,  and  speed  loaded  activity  (Fleishman  and  Quaintance,  1984) 
multiplied  by  the  amount  of  time  that  the  tasks  are  being  performed.  By  establishing  PSF 
definitions  for  modifying  the  task  time  performance  in  CoHOST  a  set  of  user  defined  PSF 
factors  can  be  defined  that  will  cause  the  amount  of  time  required  to  perform  tasks  across  the 
CoHOST  model  to  change  accordingly.  The  CoHOST  simulation  work  and  task  load 
performance  then  becomes  responsive  to  C2  system  mission  task  demands  as  they  result  in 
subsequent  operator  task  time  performance. 

Logically,  the  IVs  in  the  JNNS  regression  equations  are  independent  measures  whose 
activity  level  was  detennined  from  the  field  observations  of  TOCs.  Some  of  the  IVs,  such  as 
specific  events  that  were  either  occurring  or  not  occurring  at  the  time  of  the  observation,  were 
recorded  as  binary.  Other  variables  were  observed  on  an  interval  scale  of  1-10  indicating  level 
of  usage,  such  as  level  of  usage  of  a  communications  system  or  intelligence  collection  asset. 
Environmental  variables  were  recorded  directly  in  their  scale  of  observation  such  as  temperature 
in  degrees  F  and  percent  relative  humidity.  All  of  the  non-binary  variables  were  standardized 
into  a  range  of  0  to  1  for  the  analysis.  The  presumption  is  that  the  IVs  represent  subsystems 
whose  efficient  use  are  designed  to  enhance  the  operation  of  the  TOC.  The  efficient  use  of  these 
subsystems  do  this  by  allowing  the  operator  to  perform  more  tasks  over  time.  The  continuous 
work  and  task  load  remain  fairly  constant  over  time  but  the  task  based  parameters  such  as  the 
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number  of  tasks  queued,  dropped,  interrupted,  and  suspended  by  individual  operators  goes  down. 
As  a  result,  the  operator  continues  to  work  at  a  steady  rate  but  is  able  to  accomplish  more. 

The  process  is  to  detennine  regressor  values  from  the  JNNS  results  that  will  provide  DV 
values  at  selected  points  in  the  response  range  of  each  regression  equation.  The  DV  product 
points  were  selected  as  0%,  25%,  50%,  and  75%  as  providing  representative  points  for  the 
response  of  the  DV.  The  previous  JNNS  output  analysis  provided  17  mathematical  performance 
models  for  the  17  DV  mission  tasks  as  shown  again  in  Table  23.  The  I  Vs  used  across  all  17 
models  are  listed  in  Table  24. 

Table  23  -  JNNS  Regression  Models  /  DVs 


DV  Label 

Dependent  Variable  Description 

TP  10 

#  1  -  Primary  TasklO.  Destruction,  capture,  or  bypass  of  enemy  force. 

TP1 1 

#  2  -  Primary  Taskl  1.  Fixing  enemy  in  position. 

TP12 

#  3  -  Primary  Taskl2.  Synchronization  with  supporting  forces. 

TP  15 

#  4  -  Primary  Taskl5.  Destruction  of  first  echelon  forces. 

TP21 

#  5  -  Primary  Task21.  Peacekeeping  operations. 

TP30 

#  6  -  Primary  Task23.  Planning. 

TS12 

#  7  -  Secondary  Taskl2.  Synchronization  with  supporting  forces. 

TS18 

#  8  -  Secondary  Taskl8.  Deception  activities. 

TS19 

#  9  -  Secondary  Taskl9.  Rear  operations. 

TS30 

#  10  -  Secondary  Task23.  Planning. 

TA3 

#11-  Tertiary  Task3.  Movement  to  the  line  of  departure. 

TA21 

#  12  -  Tertiary  Task21.  Peacekeeping  operations. 

TA30 

#  13  -  Tertiary  Task23.  Planning. 

Mission  1 

#  14  -  Mission  1.  Pre  Operations  Planning. 

Mission2 

#  15  -  Mission2.  Movement  to  Contact. 

Mission3 

#  16  -  Mission3.  Attack. 

Mission5 

#  17  -  Mission5.  River  Crossing. 

Table  24  -  JNNS  Model  Regressors  / 1  Vs 


IV  Label 

IV  Description 

BOSIntell 

#1  -  Intel  BOS-  1  Conduct  Intelligence  Planning 

BOSIntel2 

#2  -  Intel  BOS-  2  Collect  Information 

BOSIntel3 

#3  -  Intel  BOS-  3  Process  Information 

BOSIntel4 

#4  -  Intel  BOS-  4  Disseminate  Intelligence 

BOSManl 

#5  -  Maneuver  BOS-  1  Conduct  Tactical  Movement 

BOSMan2 

#6  -  Maneuver  BOS-  2  Engage  Enemy  with  Direct  Fire  and  Maneuver 

BOSFS2 

#8  -  Fire  Support  BOS-  2  Employ  Field  Artillery 

BOSFS7 

#13  -  Fire  Support  BOS-  7  Conduct  Counter  Target  Acquisition  Operations 
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IV  Label 

IV  Description 

BOSFS9 

#15  -  Fire  Support  BOS-  9  Coordinate,  Synchronize,  and  Integrate  Fire  Support 

BOSMAS1 

#16  -  Mobility  &  Survivability  BOS-  1  Overcome  Obstacles 

BOSMAS2 

#17  -  Mobility  &  Survivability  BOS-  2  Enhance  Movement 

BOSMAS4 

#19  -  Mobility  &  Survivability  BOS-  4  Enhance  Physical  Protection 

BOSMAS5 

#20  -  Mobility  &  Survivability  BOS-  5  Provide  Operations  Security 

BOSMAS6 

#21  -  Mobility  &  Survivability  BOS-  6  Conduct  Deception 

BOSMAS7 

#22  -  Mobility  &  Survivability  BOS-  7  Conduct  NBC  Defense 

BOSADA1 

#23  -  Air  Defense  BOS-  1  Take  Active  Air  Defense  Measures 

BOSADA2 

#24  -  Air  Defense  BOS-  2  Take  Passive  Air  Defense  Measures 

BOSCC1 

#25  -  Command  and  Control  BOS-  1  Plan  for  Combat  Operations 

BOSCC2 

#26  -  Command  and  Control  BOS-  2  Direct  and  Lead  Unit  During  the  Preparation 
Phase  of  the  Battle 

BOSCC3 

#27  -  Command  and  Control  BOS-  3  Direct  and  Lead  Unit  in  Execution  of  Battle 

BRIntel 

#28  -  BOS  Ratings-  Intelligence 

BRMAS 

#31  -  BOS  Ratings-  Mobility  and  Survivability 

COMAFATDS 

#35  -  Communications-  AFATDS  -  Advanced  Field  Artillery  Tactical  Data  System 

COMASAS 

#37  -  Communications-  AS  AS  -  All  Source  Analysis  System 

COMMCS 

#43  -  Communications-  MCS  -  Maneuver  Control  System  (Heavy  &  Light) 

COMAMDWS 

#44  -  Communications-  AMDWS.  ADA 

INTELTUAV 

#47  -  Intel  Collection-  TUAV  -  Tactical  Unmanned  Aerial  Vehicle 

MISCBattleTiming 

#50  -  Misc  Factors-  Battle  Timing 

MISCBattleT  empo 

#51  -  Misc  Factors-  Battle  Tempo 

MISCReconOps 

#52  -  Misc  Factors-  Reconnaissance  Operations 

MISCInfoOps 

#53  -  Misc  Factors-  Information  Operations 

MISCTOCActivity 

#55  -  Misc  Factors-  Observed  Activity  /  Stress  Level  in  TOC 

MOPPLevel 

#56  -  Misc  Factors-  MOPP  Level 

StaffHud 

#57  -  Misc  Factors-  Staff  Huddle 

PERSCMDR 

#58  -  Personnel-  Commanding  Officer  present  in  TOC 

PERSXO 

#59  -  Personnel-  Executive  Officer  present  in  TOC 

PERSBC 

#60  -  Personnel-  Battle  Captain  present  in  TOC 

PERSS3 

#61  -  Personnel-  S3  present  in  TOC 

PERSS3NCO 

#62  -  Personnel-  S3  NCO  present  in  TOC 

PERSS3RTO 

#63  -  Personnel-  S3  RTO  present  in  TOC 

PERSS2 

#64  -  Personnel-  S2  present  in  TOC 

PERSFSO 

#65  -  Personnel-  FSO  present  in  TOC 

PERSENGR 

#66  -  Personnel-  Engineer  present  in  TOC 

PERSALO 

#67  -  Personnel-  ALO  present  in  TOC 

PERSS3SGM 

#68  -  Personnel-  S3  SGM  present  in  TOC 

Flumidity 

#69  -  Environmental  Factor-  Relative  Humidity,  % 

The  steps  to  determine  the  regressor  values  required  to  produce  the  desired  DV  responses 
for  each  of  the  17  perfonnance  models  are: 

•  Set  all  binary  IVs  (regressors)  to  1,  indicating  that  they  are  in  play  or  in  use,  etc,  since 
they  were  observed  as  binary  active  or  not  active  variables. 

•  Set  all  real  IVs  (regressors)  also  to  1  indicating  that  they  are  at  the  full  range  of  their 
performance. 
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•  Calculate  DV  from  this  setting,  which  becomes  the  MAX  range  value. 

•  Recognizing  that  the  intercept  is  the  value  when  all  IVs  are  set  to  zero,  set  the  MIN  range 
value  to  the  value  of  the  intercept. 

•  Calculate  the  RANGE  of  the  DV  response  =  MAX  -  MIN. 

•  Calculate  the  MEDIAN  of  the  range  as  the  mid  point  in  the  RANGE. 

•  Determine  the  DV  Levels  in  the  range  of  25%,  50%,  75%: 

o  Calculate  the  quartile  of  the  range,  QTR  =  RANGE  /  4. 
o  Calculate  the  25%  point  in  the  range  =  MEDIAN  -  QTR. 
o  Calculate  the  50%  point  in  the  range  =  MEDIAN, 
o  Calculate  the  75%  point  in  the  range  =  MEDIAN  +  QTR. 

•  Calculate  the  DV  Target  Value  for  each  of  the  3  DV  Levels  of  25%,  50%,  and  75%  by: 

o  Iteratively  adjust  the  IV  values  in  each  DV  regression  equation  to  one  of  their 
two  possible  binary  values,  and  the  real  values  from  0  to  1  until  a  combination  of 
the  IVs  is  reached  where  the  regression  equation  result  provides  a  value 
approximating  as  closely  as  possible  the  25%,  50%,  and  75%  levels  determined 
from  the  RANGE  of  the  DV. 

o  The  resulting  set  of  IVs  identified  as  TRUE,  or  with  a  value  greater  than  zero 
indicate  those  IVs  that  are  predicted  to  be  significant  in  the  performance  of  that 
DV  at  each  of  the  respective  treatment  levels. 

The  calculations  to  determine  the  DV  Target  Values  for  the  25%,  50%,  and  75% 

utilization  levels  is  at  Table  25  and  a  sample  of  the  corresponding  IV  matrix  for  the  DV  model 

TP10  is  at  Table  26.  A  complete  table  of  all  the  DV  /  IV  correlation  matrices  for  each  of  the  17 

DV  models  is  at  Appendix  P.  Listed  beside  the  IV  matrix  in  Table  25  are  those  regressors  that 

were  required  to  make  the  regression  equation  provide  the  required  DV  value  to  correspond  with 

each  of  the  3  utilization  levels.  These  regressors  then  become  the  factors  that  are  identified  as 

those  perfonnance  tasks  that  are  predicted  to  significantly  impact  the  mission  task  at  that 

utilization  level. 
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Table  25  -  Determination  of  DV  Target  Values 


DV 

Target 

Value 

DV 

Stats 

DV 

Levels 

DV 

Label 

0.25 

0.5 

0.75 

MAX 

MIN 

Range 

Qtr= 
Range 
/  4 

Median 

0.25 

0.5 

0.75 

TP10 

0.0978 

0.1236 

0.1236 

0.1236 

0.0978 

0.0258 

0.0065 

0.1107 

0.1043 

0.1107 

0.1172 

TP11 

0.2042 

0.7804 

1.2037 

1.7186 

-0.2610 

1.9796 

0.4949 

0.7288 

0.2339 

0.7288 

1.2237 

TP12 

-0.4774 

-0.0857 

0.5515 

0.9788 

-0.8666 

1.8454 

0.4614 

0.0561 

-0.4053 

0.0561 

0.5175 

TP15 

-0.3028 

0.0022 

0.3139 

0.6282 

-0.6194 

1.2476 

0.3119 

0.0044 

-0.3075 

0.0044 

0.3163 

TP21 

-0.5036 

-0.0440 

0.4460 

0.4755 

-0.5332 

1.0087 

0.2522 

-0.0288 

-0.2810 

-0.0288 

0.2234 

TP23 

0.9153 

1.6095 

1.9265 

2.6450 

0.3903 

2.2547 

0.5637 

1.5177 

0.9540 

1.5177 

2.0813 

TS12 

-0.0436 

-0.0436 

-0.0436 

-0.0436 

0.0412 

-0.0848 

-0.0212 

-0.0012 

0.0200 

-0.0012 

-0.0224 

TS18 

0.1338 

0.2935 

0.4047 

0.5595 

-0.0209 

0.5804 

0.1451 

0.2693 

0.1242 

0.2693 

0.4144 

TS19 

0.9781 

0.9781 

0.9781 

0.9781 

0.0989 

0.8791 

0.2198 

0.5385 

0.3187 

0.5385 

0.7583 

TS23 

-0.3058 

0.0886 

0.2260 

0.4519 

-0.5006 

0.9525 

0.2381 

-0.0243 

-0.2624 

-0.0243 

0.2138 

TA3 

0.3078 

0.4981 

0.5768 

0.8565 

0.0898 

0.7667 

0.1917 

0.4731 

0.2815 

0.4731 

0.6648 

TA21 

-0.2860 

-0.2163 

-0.1712 

-0.1015 

-0.3323 

0.2308 

0.0577 

-0.2169 

-0.2746 

-0.2169 

-0.1592 

TA23 

0.8937 

1.1810 

2.4907 

3.3385 

-0.0603 

3.3989 

0.8497 

1.6391 

0.7894 

1.6391 

Missionl 

-0.0886 

-0.0886 

0.1815 

0.3222 

-0.2187 

0.5409 

0.1352 

0.0517 

-0.0835 

0.0517 

0.1869 

Mission2 

0.2245 

0.2256 

0.3112 

0.3112 

0.2142 

0.0970 

0.0243 

0.2627 

0.2384 

0.2627 

0.2869 

Mission3 

0.0426 

0.3974 

0.4620 

0.8169 

-0.1561 

0.9730 

0.2432 

0.3304 

0.0871 

0.3304 

0.5736 

Mission5 

-0.1868 

0.1075 

0.4407 

0.9907 

-0.6554 

1.6461 

0.4115 

0.1676 

-0.2439 

0.1676 

0.5792 

Table  26  -  Sample  DV  /  IV  Correlation  Matrix  For  Task  TP10 


] 

Model  #  1 

-  Primary  TasklO.  Destruction,  capture,  or  bypass  of  enemy  force. 

Variable 

Inter 

cept 

BOS 

Intel4 

BOS 

MAS2 

BOS 

CC2 

MOPP 

Level 

All  I Vs 

1.0 

1 

1 

1 

1 

All  I Vs 

0.0 

0 

0 

0 

0 

DV  .25 

0 

0 

0 

0 

No  I Vs 

DV  0.5 

1 

1 

1 

1 

BOSIntel4,  BOSMAS2,  BOSCC2, 
MOPPLevel 

DV  .75 

1 

1 

1 

1 

BOSIntel4,  BOSMAS2,  BOSCC2, 
MOPPLevel 

Beta 

Weight 

0.09781 

0.37701 

0.31685 

-0.19909 

-0.46896 
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4.1.2.  CoHOST  Simulation. 


CoHOST  is  a  discrete  event  computer  simulation  of  TOC  operations  developed  by  ARL- 
HRED  in  support  of  various  Army  studies  evaluating  current  and  proposed  TOC  configurations. 
It  models  23  members  of  the  battalion  or  brigade  command  staff  and  tracks  workload  /  taskload 
and  utilization  over  time  for  each  of  the  23  operators.  It  also  tracks  task  activity  for  each 
operator  that  includes  the  number  of  tasks  queued,  dropped,  interrupted,  and  suspended  for  each 
of  the  23  operators.  The  original  project  developed  two  scenarios  for  use  with  this  simulation. 
The  first  was  a  four  phase  combat  scenario  consisting  of  planning,  movement  to  contact,  attack, 
and  defense  phases  occurring  over  an  approximate  8  hour  period.  The  second  scenario  was  a 
long  movement  Desert  Storm  like  scenario  where  the  friendly  forces  conducted  a  movement  to 
contact  tactical  movement  across  approximately  180  kilometers  spanning  24  hours  before 
making  contact  with  the  enemy. 

The  CoHOST  simulation  was  executed  in  the  Microsoft  Windows™  environment  running 
the  Micro  Saint™  programming  language.  Micro  Saint™  is  a  discrete  event  simulation  language 
that  is  designed  to  support  human  perfonnance  studies  and  is  applicable  for  a  wide  range  of 
human  performance  domains  (Laughery  and  Corker,  1997).  The  version  used  here  was  Release 
3.1  Build  A  with  ActionView  and  OptQuest,  Standard  Version,  that  was  released  on  October  27, 
1999  and  operates  in  a  Windows  98  /  NT  /  2000  /  XP  environment. 

The  CoHOST  simulation  model  was  configured  to  execute  with  the  PSF  set  to  represent 
the  effects  of  each  of  the  4  values  of  the  DV  response  of  0%,  25%,  50%,  and  75%.  As  higher 
DV  response  utilizations  represent  more  efficient  operation  while  higher  PSF  settings  represent 
longer  task  times  (i.e.,  less  efficient  operation),  an  inverse  relationship  exists  between  the  DV 
utilization  percentiles  and  the  PSF  settings.  For  the  purposes  of  these  developmental  runs  a  SME 
estimation  was  made  that  presumed  a  75%  DV  utilization  equated  to  a  30%  decrease  in  task 
times  resulting  from  the  PSF  setting  (The  value  of  30%  was  subjectively  chosen  based  upon 
individual  observation  of  field  TOC  perfonnance.  For  the  purposes  of  this  exploratory  process 
development  this  is  deemed  a  satisfactory  reference  point  upon  which  to  build  the  analytical 
technique.  Future  research  will  validate  and  /  or  update  this  value  with  further  SME 
interrogation).  Thus,  each  of  the  DV  utilization  levels  resulted  in  PSF  settings  as  shown  in  Table 
27. 
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Table  27  -  DV  Utilization  /  PSF  Setting  Correlation 


•  0%  DV  utilization  ==  PSF  =  0  (This  is  the  condition  where  all  the  IVs  are 

set  to  zero,  i.e.,  turned  off  or  not  in  play,  and  the  numerical  value  of  the  DV 
becomes  the  regression  intercept.) 

•  25%  DV  utilization  ==  PSF  =  -10. 

•  50%  DV  utilization  ==  PSF  =  -20. 

•  75%  DV  utilization  ==  PSF  =  -30. 


With  stochastic  simulations  such  as  CoHOST,  multiple  replications  of  the  simulation 
must  be  performed  to  control  the  effects  of  random  variability  in  the  model.  In  this  study  the 
desire  was  to  have  the  output  data  exhibit  a  95%  probability  of  falling  within  the  confidence  limit 
which  gives  a  specified  error  level,  s,  equal  to  ±.05  of  the  mean  for  each  data  element.  Using 
Banks’  procedures  (Banks  et  ah,  1996),  the  number  of  replications  required  for  each  simulation 
run  to  achieve  this  desired  error  level  was  determined  to  be  15  (see  Appendix  Q).  Each  of  the 
simulation  conditions  was  run  for  this  number  of  replications. 

Prior  to  running  the  simulation,  the  scenario  start  and  stop  times  for  each  of  the  combat 
phases  was  determined  in  order  to  apply  the  appropriate  DV  model  to  the  combat  phase  for 
which  it  applies.  The  four  combat  phases  in  the  scenario  are  (1)  Pre-Operations  Planning,  (2) 
Movement  to  Contact,  (3)  Attack,  and  (4)  Defense.  The  number  of  simulation  runs  was  therefore 
one  combined  run  for  each  PSF  setting  where  each  combined  run  contained  the  four  scenario 
phases  for  a  total  of  four  simulation  runs  with  each  run  consisting  of  15  replications. 

The  PSF  functions  are  established  and  the  CoHOST  simulations  executed  by: 

•  Use  Banks’  replication  analysis  (Banks  et  ah,  1996)  to  determine  the  number  of 
replications  to  run  the  CoHOST  simulation  for  each  treatment  condition,  R,  to 
account  for  random  variability  in  the  simulation.  In  this  case,  this  is  established 
as  15  (See  Appendix  Q). 

•  Modify  the  CoHOST  task  parameter  definition  arrays  in  the  model  to  reflect  the 
desired  PSF  settings  of  0  (default),  -10,  -20,  and  -30,  meaning  to  reduce  the  task 
times  by  0%,  10%,  20%,  and  30%  as  the  different  treatment  conditions  to  be 
evaluated. 

•  Execute  each  of  the  resulting  four  simulation  models  with  all  output  turned  on  and 
R  set  equal  to  15.  Also,  set  the  initial  random  number  seed  equal  to  one,  ”1,”  at 
the  beginning  of  each  treatment  run  for  consistency. 
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4.2.  Results. 


The  mathematical  models  developed  in  Phase  I  were  applied  as  algorithmic  predictors 
used  during  the  execution  of  the  CoHOST  TOC  simulation.  Interface  characteristics  to  translate 
the  mission  based  models  from  the  JNNS  to  the  task  and  workload  constructs  existent  in  the 
CoHOST  model  were  developed  that  allow  the  translation  of  the  mission  based  parameters  that 
were  recorded  during  the  naturalistic  observations  in  the  TOC  to  the  human  operator  task  and 
workload  characteristics  that  are  modeled  in  CoHOST.  Using  these  interfaces,  CoHOST  was 
executed  to  produce  results  that  provided  the  basis  for  discussion  and  analysis. 

4.2. 1 .  Evaluation  Of  the  C2S  With  CoHOST  Discrete  Event  Computer  Simulation. 

The  CoHOST  run  results  are  analyzed  to  make  predictions  as  to  which  IV  performance 
elements  make  contributions  during  various  types  of  combat  actions  to  improve  overall  operator 
performance.  This  is  one  of  many  possible  approaches  for  trying  to  optimize  the  performance  of 
a  system  through  computer  simulation.  The  presumption  is  that  a  decrease  in  task  performance 
time  will  result  in  more  efficient  operation  of  the  TOC  along  with  the  ability  to  achieve  better 
situational  awareness  of  the  battlefield  and  allow  the  commander  an  enhanced  ability  to  make 
better  decisions.  This  is  achieved  by  allowing  the  operator  to  be  more  effectively  utilized  with  a 
greater  workload  throughput  performance  as  a  result  of  less  time  being  required  to  perfonn 
individual  tasks.  The  application  of  the  COMPASS  paradigm  allows  hypotheses  to  be  generated 
regarding  the  overall  performance  of  the  C2S.  One  such  hypothesis  regarding  the  overall 
performance  of  the  system  is: 

Hypothesis:  Decreasing  the  amount  of  time  required  to  perform  individual  tasks  in  a 
TOC  based  work  environment  results  in  more  effective  operator  performance  through  better 
utilization  of  the  operator ’s  time  by  providing  greater  workload  throughput  performance 
because  of  the  ability  to  perform  more  tasks  over  the  same  amoun  t  of  time. 

The  COMPASS  paradigm  allows  a  relationship  of  overall  task  based  operator  workload 
to  be  tested  against  those  mission,  hardware,  and  software  task  components  that  can  affect 
operator  performance  during  combat  activities  in  the  TOC. 
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4.2.1 . 1 .  CoHOST  Data  Reduction. 


Each  CoHOST  simulation  run  produces  a  number  of  ASCII  text  results  files  with  an 
extension  of  .RES.  There  is  one  file  containing  workload  and  utilization  data  for  each  operator, 
one  file  containing  task  queue,  dropped,  and  interrupted  data  for  all  operators,  and  one  file 
containing  summarized  utilization  infonnation  for  all  operators.  The  general  procedure  to  first 
reduce  the  operator  utilization  and  workload  data  for  analysis  is: 

•  Load  the  individual  data  for  each  operator  into  a  separate  spreadsheet  and  then 
combine  all  the  operator  spreadsheets  into  one. 

•  Sort  the  combined  spreadsheet  by  replication  number,  clock  time,  and  operator  id. 

•  Extract  the  individual  components  for  operator  id,  clock  time,  utilization,  and 
overall  workload  into  separate  spreadsheets  for  each  replication. 

•  Determine  the  mean  for  the  clock  time,  utilization,  and  overall  workload  for  the 
15  replication  runs. 

•  As  the  operator  data  is  categorical  it  cannot  be  averaged.  Determine  a 
representative  replication  from  the  1 5  for  the  operator  id  and  use  it  with  the 
subsequent  analysis  along  with  the  rest  of  the  data. 

•  Paste  the  data  for  the  selected  operator  ID  replication,  and  the  mean  values  for  the 
clock  time,  utilization,  and  workload  into  a  new  spreadsheet. 

•  Sort  the  new  worksheet  by  the  operator  ID  to  block  the  data  into  performance 
blocks  for  each  operator. 

•  For  each  operator  to  be  analyzed,  plot  the  utilization  and  workload  data  by  time 
for  each  of  the  four  simulation  runs  for  a  total  of  8  plots  for  each  operator.  Add  a 
linear  regression  trendline  onto  each  plot  along  with  the  equation  of  the  line  for 
analysis  and  to  more  effectively  represent  the  difference  between  the  plot  for  each 
treatment.  A  sample  of  these  plots  for  operator  21,  Fire  Support  Officer,  is  at 
Figures  35  for  utilization,  and  36  for  workload. 


4.2.1 .2.  CoHOST  Data  Analysis. 

Figures  35  and  36  show  the  workload  and  utilization  profiles  over  time  for  the  Fire 
Support  Officer.  The  four  phases  in  the  simulation  scenario,  preoperations  planning,  movement 
to  contact  (MTC),  hasty  attack,  and  defense  in  sector,  are  delineated  on  each  of  the  plots.  A 
linear  regression  trendline  is  also  shown  on  each  plot  along  with  its  equation  under  the  legend. 
The  first  point  to  note  in  examining  these  plots  is  that  overall  utilization  remains  roughly 
constant,  although  with  a  slight  increase,  as  the  task  times  are  reduced  from  0%  to  10%,  to  20%, 
and  30%.  Workload  has  a  slight  increase  initially  at  10%  and  then  stabilizes  becoming  almost 
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identical  at  20%  and  30%.  The  initial  conclusion,  from  only  regarding  these  graphs,  is  that 
decreasing  the  overall  times  to  perfonn  tasks  has  little  effect  and  might  actually  increase 
workload  and  utilization  slightly. 
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•  Operator  task  performance  data: 

o  Load  the  task  data  results  file  from  each  of  the  four  simulation  runs  into  a 
spreadsheet. 

o  Tabulate  the  data  for  number  of  task  queues,  number  of  task  interrupts, 
number  of  task  suspensions,  and  number  of  task  drops  against  the  four 
simulation  runs  for  each  operator. 

o  Plot  the  four  task  conditions  over  the  four  simulation  runs  for  each 
operator  to  provide  a  task  performance  profile  for  each  operator.  A 
sample  plot  of  these  task  conditions  for  operator  12,  Battle  Captain,  is  at 
Figure  37  and  for  operator  21,  Fire  Support  Officer  at  Figure  38.  The 
horizontal  axis  indicates  the  results  from  the  four  simulation  runs  where 
the  MOO  point  indicates  the  baseline  run  with  “0”  reduction  in  task  time 
performance,  the  M10  point  represents  the  run  with  a  10%  reduction  in 
task  time  performance,  the  M20  point  represents  the  run  with  a  20% 
reduction  and  the  M30  point  represents  the  run  with  a  30%  reduction. 
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Figure  37  -  Task  Performance  for  Operator  12,  Battle  Captain 
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Figure  38  -  Task  Performance  for  Operator  21,  Fire  Support  Officer 

However,  when  these  observations  are  combined  with  the  task  performance  results  shown 
in  Figure  35,  a  more  complete  picture  starts  to  emerge.  With  shorter  task  performance  times 
more  tasks  can  be  performed  over  the  same  period  of  time.  Similar  results  are  shown  in  Figure 
36  for  the  Battle  Captain.  If  an  operator  finishes  a  task  early  or  more  quickly,  that  operator  is  not 
going  to  just  wait  idle  until  the  next  task  arrives  thereby  achieving  a  lower  workload. 

Considering  the  mission-oriented  focus  of  the  TOC  work  group,  operators  will  immediately 
proceed  on  to  the  next  task  when  the  current  perfonnance  effort  is  completed.  This  is  reflected 
in  the  utilization  and  workload  performance  profiles  in  Figures  37  and  38. 

On  the  other  hand,  this  trend  can  only  continue  up  to  a  point.  This  condition  is  where  the 
operator  becomes  physically  and  /  or  cognitively  saturated  and  can  no  longer  keep  pace  with 
increasing  performance  demands  resulting  from  quicker  task  execution.  This  condition  is 
reflected  in  the  charts  in  Figures  37  and  38  in  the  Total  Summary  chart  for  the  Battle  Captain  and 
Fire  Support  Officer,  which  shows  the  number  of  tasks  that  were  queued,  interrupted,  suspended, 
and  dropped  during  the  overall  scenario.  When  task  performance  times  are  reduced  from  10%  to 
20%,  while  the  utilization  and  workload  profiles  remain  fairly  constant,  the  task  performance 
increases  with  fewer  numbers  of  tasks  that  are  queued  up,  interrupted,  suspended  and  dropped. 
This  predicts  increased  operator  effectiveness  because  more  tasks  can  be  performed  because  they 
take  less  time  to  execute  as  long  as  the  operator  has  the  psychophysiological  ability  to  keep  up 
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with  the  increased  task  demand.  But,  with  further  reductions  in  task  time  requirements  that  limit 
can  be  exceeded.  Between  a  task  time  reduction  of  20%  to  30%  the  Total  Summary  chart  in 
Figure  38  shows  that  the  operator  is  apparently  no  longer  able  to  keep  up  with  increased  task 
demands  with  the  result  that  task  performance  begins  to  degrade  with  increases  in  the  number  of 
queues,  interrupts,  suspended  tasks,  and  dropped  tasks.  The  conclusion  is  that  the  tasks 
performance  profiles  between  10%  to  30%  reflect  that  an  operator  overload  condition  has 
occurred  somewhere  along  this  perfonnance  period  causing  a  transition  from  peak  to  degraded 
operator  perfonnance. 

4.2.2.  Predictions  of  TOC  Activity  From  Mapping  CoHOST  Results  to  JNNS  Predictors. 

The  final  stage  of  this  analysis  is  to  relate  the  predictors  from  the  JNNS  neural  network 
simulation  to  performance  parameters  in  the  CoHOST  simulation.  Estimates  can  then  be  made 
as  to  the  causal  relationships  between  the  factors  influencing  TOC  performance  at  the  system 
level  as  represented  by  JNNS  and  human  perfonnance  in  response  to  task  stimuli  as  represented 
in  CoHOST.  These  associations  can  then  be  interpreted  as  to  the  resulting  impact  on  human 
performance  that  might  result  from  proposed  changes  to  TOC  systems. 

The  relationship  between  task  time  decrease  in  CoHOST  and  the  DV  response  by  the 
regression  models  from  JNNS,  as  stated  previously,  was  determined  from  applying  the  subjective 
experience  of  one  observer  from  working  in  and  observing  TOC  operations.  While  the  level  of 
accuracy  of  the  numbers  may  be  in  need  of  continued  refinement,  the  relationship  that  they 
represent  is  not.  The  COMPASS  paradigm  states  that  there  is  a  relationship  between  DV 
utilization  levels  in  the  TOC  and  overall  resulting  task  performance  time  that  this  imposes  on  the 
operators.  The  exact  level  of  the  PSF  that  corresponds  to  utilization  levels  of  the  various  DVs  is 
a  subject  for  continued  field  observation  and  simulation  testing  and  evaluation.  However,  the 
use  of  initial  subjective  opinion  for  test  parameters  provides  the  basis  for  the  examination  of  the 
causal  relationships  that  exist  here  and  allows  the  development  of  procedures  to  systematically 
examine  them. 

The  DV  /  IV  correlation  matrices  at  Table  26  and  Appendix  P  show  IV  from  the  JNNS 
simulation  that  have  been  determined  to  have  some  significance  at  various  stages  of  the  scenario 
as  overall  task  time  decreases.  Comparing  these  predictors  to  the  CoHOST  simulation  results 
such  as  those  presented  in  Figures  35-38  allows  this  analysis  to  occur. 
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This  analysis  will  focus  on  operator  #21,  the  Fire  Support  Officer  (FSO),  and  operator 
#12,  the  Battle  Captain  (BC).  Both  of  these  operators  play  primary  roles  in  the  TOC  and  both 
are  critical  to  the  overall  success  of  the  TOC  operation.  Their  taskload  and  utilization  profiles 
are  representative  of  the  type  of  activity  expected  by  operators  whose  job  is  to  work  at  high 
levels  of  coordination  and  interaction  with  other  operators.  It  is  the  quality  of  efforts  by 
individuals  such  as  these  that  will  determine  how  good  the  conditions  are  upon  which  the  actual 
decision  makers  base  their  judgments. 

The  role  of  the  FSO  is  to  act  as  the  TOC  single  point  of  contact  for  directing  and 
coordinating  supporting  cannon  and  missile  artillery  during  combat.  This  operator  has  dedicated 
voice  and  digital  communications  links  with  fire  direction  control  centers  and  firing  batteries  and 
is  the  one  who  initiates  calls  for  fire  missions  based  on  the  decisions  made  in  the  TOC.  The  FSO 
is  responsible  for  carrying  out  the  priority  allocation  of  artillery  missions  as  directed  by  the 
commander  and  for  seeing  that  all  requests  for  fire  support  are  directed  to  the  appropriate 
receiver. 

The  role  of  the  BC  can  be  compared  to  that  of  an  office  manager  in  the  civilian  world. 

The  BC  coordinates  the  minute-to-minute  activities  of  the  various  sections  in  the  TOC  and 
insures  that  all  the  commander’s  policies,  procedures,  guidelines,  and  directives  are  implemented 
as  ordered  when  they  are  required.  When  neither  the  commander,  executive  officer,  or  S3 
operations  officer  are  present,  the  BC  is  in  command  of  the  TOC. 

4.2.2. 1.  Scenario  Phase  1  -  Preoperations  Planning. 

This  phase  of  the  scenario  consists  of  moving  combat  forces  forward  to  the  line  of 
departure  (LD)  and  final  preparation  planning  of  combat  activities  immediately  prior  to  the 
commencement  of  actual  combat.  During  this  phase  the  CoHOST  simulation  represents  the  FSO 
as  being  fairly  busy  with  a  constant  high  utilization  rate  and  constant  workload  (previously 
shown  in  Figures  35  and  36).  As  task  times  are  decreased  from  0%  to  30%  below  the  default, 
the  task  performance  of  the  FSO  drops  across  all  categories  of  task  performance  (number  of  task 
queues,  number  of  interrupted  tasks,  number  of  suspended  tasks,  and  number  of  dropped  tasks  as 
shown  in  Figure  38). 

•  Table  28  shows  those  tasks,  predicted  by  JNNS  at  the  -10%  PSF  level,  during  this 
tactical  phase  whose  increased  utilization  can  cause  these  task  time  reduction 
effects.  At  the  -10%  PSF  level  this  includes  such  things  as  the  requirement  to 
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coordinate  and  integrate  fire  support.  A  way  to  do  this  is  by  more  effective 
utilization  of  the  artillery  digital  fire  communications  system  AFATDS  or  by 
more  effective  design  of  this  system.  The  presence  of  the  ALO  in  the  TOC  during 
this  time  can  also  improve  overall  performance  as  the  ALO  is  also  responsible  for 
air  supported  fire  missions  and  works  closely  with  the  FSO. 

Table  28  -  DV  /  IV  Correlation  -  Planning  -  PSF  @  -10% 


Phase  1-  Planning  (DV  @  0.25,  -10%  PSF) 


Dependent  Variable  /  Model 

Independent  Variable 

Multi¬ 

plication 

Factor 

#  05  -  Primary  Task2 1 . 
Peacekeeping 
operations. 

#51  -  Misc  Factors-  Battle  Tempo 

1 

#  06  -  Primary  Task23. 

Planning 

#28  -  BOS  Ratings-  Intelligence 

2 

#  06  -  Primary  Task23. 

Planning 

#57  -  Misc  Factors-  Staff  Fluddle 

2 

#  08  -  Secondary  Taskl8. 

Deception  activities. 

#25  -  Command  and  Control  BOS-  1  Plan 
for  Combat  Operations 

1 

#  09  -  Secondary  Taskl9. 

Rear  operations. 

#21  -  Mobility  &  Survivability  BOS-  6 
Conduct  Deception 

1 

#  09  -  Secondary  Taskl9. 

Rear  operations. 

#64  -  Personnel-  S2  present  in  TOC 

1 

#  10  -  Secondary  Task23. 
Planning 

#02  -  Intel  BOS-  2  Collect  Information 

1 

#  10  -  Secondary  Task23. 
Planning 

#03  -  Intel  BOS-  3  Process  Information 

1 

#  10  -  Secondary  Task23. 
Planning 

#15  -  Fire  Support  BOS-  9  Coordinate, 
Synchronize,  and  Integrate  F  ire 
Support 

2 

#  12  -  Tertiary  Task21. 
Peacekeeping 
operations. 

#55  -  Misc  Factors-  Observed  Activity  / 
Stress  Level  in  TOC 

1 

#  12  -  Tertiary  Task21. 
Peacekeeping 
operations. 

#67  -  Personnel-  ALO  present  in  TOC 

1 

#  13  -  Tertiary  Task23. 

Planning 

#01  -  Intel  BOS-  1  Conduct  Intelligence 
Planning 

1 

#  13  -  Tertiary  Task23. 

Planning 

#50  -  Misc  Factors-  Battle  Timing 

1 

#  14  -  Mission  1.  Pre 

Operations  Planning 

#16  -  Mobility  &  Survivability  BOS-  1 
Overcome  Obstacles 

1 

•  Table  29  shows  those  tasks,  predicted  by  JNNS  at  the  -20%  PSF  level,  during  this 
tactical  phase  whose  increased  utilization  can  cause  these  task  time  reduction 
effects.  Decreasing  the  amount  of  time  to  perform  tasks  20%  below  the  default 
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focuses  on  tasks  that  improve  the  collection  of  intelligence  such  as  the  conduct  of 
information  operations,  the  presence  of  the  S2  intelligence  officer  in  the  TOC  and 
the  planning  and  collecting  of  intelligence  information.  Systems  critical  to  this 
activity  includes  the  all  source  analysis  system  (ASAS),  which  is  the  digital 
communications  system  for  this  purpose. 

Table  29  -  DV  /  IV  Correlation  -  Planning  -  PSF  @  -20% 


Phase  1-  Planning  (DV  @  0.5,  -20%  PSF) 


Dependent  Variable  /  Model 

Independent  Variable 

Multi¬ 

plication 

Factor 

#  05  -  Primary  Task2 1 . 
Peacekeeping 
operations. 

#51  -  Misc  Factors-  Battle  Tempo 

1 

#  05  -  Primary  Task2 1 . 
Peacekeeping 
operations. 

#53  -  Misc  Factors-  Information  Operations 

1 

#  06  -  Primary  Task23. 

Planning 

#21  -  Mobility  &  Survivability  BOS-  6 
Conduct  Deception 

2 

#  06  -  Primary  Task23. 

Planning 

#57  -  Misc  Factors-  Staff  Fluddle 

1 

#  08  -  Secondary  Taskl8. 

Deception  activities. 

#08  -  Fire  Support  BOS-  2  Employ  Field 
Artillery 

1 

#  08  -  Secondary  Taskl8. 

Deception  activities. 

#25  -  Command  and  Control  BOS-  1  Plan 
for  Combat  Operations 

1 

#  09  -  Secondary  Taskl9.  Rear 
operations. 

#64  -  Personnel-  S2  present  in  TOC 

1 

#  10  -  Secondary  Task23. 
Planning 

#19  -  Mobility  &  Survivability  BOS-  4 
Enhance  Physical  Protection 

1 

#  12  -  Tertiary  Task21. 
Peacekeeping 
operations. 

#55  -  Misc  Factors-  Observed  Activity  / 
Stress  Level  in  TOC 

1 

#  13  -  Tertiary  Task23. 

Planning 

#01  -  Intel  BOS-  1  Conduct  Intelligence 
Planning 

1 

#  13  -  Tertiary  Task23. 

Planning 

#02  -  Intel  BOS-  2  Collect  Information 

1 

#  14  -  Mission  1.  Pre 

Operations  Planning 

#16  -  Mobility  &  Survivability  BOS-  1 
Overcome  Obstacles 

1 

•  Table  30  shows  those  tasks,  predicted  by  JNNS  at  the  -30%  PSF  level,  during  this 
tactical  phase  whose  increased  utilization  can  cause  these  task  time  reduction 
effects.  Further  decreasing  the  overall  task  performance  time  30%  below  the 
default  can  be  propagated  by  even  more  effective  intelligence  operations  such  as 
more  effective  planning,  collecting,  processing,  and  disseminating  of  intelligence 
information.  Activities  associated  with  the  coordination  and  synchronization  of 
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fire  support  assets  also  is  critical.  Increased  message  traffic  from  both  ASAS  and 
AFATDS  can  contribute  to  the  more  effective  implementation  of  these  activities. 

Table  30  -  DV  /  IV  Correlation  -  Planning  -  PSF  @  -30% 


Phase  1-  Planning  (DV  @  0.75,  -30%  PSF) 


Dependent  Variable  /  Model 

Independent  Variable 

Multi¬ 

plication 

Factor 

#  05  -  Primary  Task21. 
Peacekeeping 
operations. 

#15  -  Fire  Support  BOS-  9  Coordinate, 
Synchronize,  and  Integrate  F  ire 
Support 

2 

#  05  -  Primary  Task21. 

Peacekeeping  ops. 

#20  -  Mobility  &  Survivability  BOS-  5 
Provide  Operations  Security 

1 

#  05  -  Primary  Task2 1 . 
Peacekeeping 
operations. 

#53  -  Misc  Factors-  Information  Operations 

2 

#  06  -  Primary  Task23. 

Planning 

#21  -  Mobility  &  Survivability  BOS-  6 
Conduct  Deception 

3 

#  06  -  Primary  Task23. 

Planning 

#28  -  BOS  Ratings-  Intelligence 

2 

#  06  -  Primary  Task23. 

Planning 

#57  -  Misc  Factors-  Staff  Fluddle 

2 

#  08  -  Secondary  Taskl8. 

Deception  activities. 

#08  -  Fire  Support  BOS-  2  Employ  Field 
Artillery 

1 

#  08  -  Secondary  Taskl8. 

Deception  activities. 

#59  -  Personnel-  Executive  Officer  present 
in  TOC 

1 

#  09  -  Secondary  Taskl9.  Rear 
operations. 

#64  -  Personnel-  S2  present  in  TOC 

1 

#  10  -  Secondary  Task23. 
Planning 

#19  -  Mobility  &  Survivability  BOS-  4 
Enhance  Physical  Protection 

1 

#  12  -  Tertiary  Task21. 
Peacekeeping 
operations. 

#67  -  Personnel-  ALO  present  in  TOC 

1 

#  13  -  Tertiary  Task23. 

Planning 

#01  -  Intel  BOS-  1  Conduct  Intelligence 
Planning 

1 

#  13  -  Tertiary  Task23. 

Planning 

#02  -  Intel  BOS-  2  Collect  Information 

1 

#  13  -  Tertiary  Task23. 

Planning 

#03  -  Intel  BOS-  3  Process  Information 

1 

#  13  -  Tertiary  Task23. 

Planning 

#04  -  Intel  BOS-  4  Disseminate 

Intelligence 

1 

#  13  -  Tertiary  Task23. 

Planning 

#50  -  Misc  Factors-  Battle  Timing 

1 

#  13  -  Tertiary  Task23. 

Planning 

#63  -  Personnel-  S3  RTO  present  in  TOC 

1 

#  14  -  Mission  1.  Pre 

Operations  Planning 

#16  -  Mobility  &  Survivability  BOS-  1 
Overcome  Obstacles 

2 

#  14  -  Mission  1.  Pre 

Operations  Planning 

#17  -  Mobility  &  Survivability  BOS-  2 
Enhance  Movement 

1 

#  14  -  Mission  1.  Pre 

Operations  Planning 

#43  -  Communications-  MCS  -  Maneuver 
Control  System  (Fleavy  &  Light) 

1 
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4.2. 2.2.  Scenario  Phase  2  -  Movement  To  Contact  (MTC). 

This  phase  of  the  scenario  involves  the  initiation  of  movement  toward  the  supposed 
location  of  the  enemy  forces.  The  line  of  departure  (LD)  is  crossed  according  to  a  time 
movement  table  and  forces  are  deployed  and  prepared  for  combat  but  no  shots  are  being  fired,  as 
there  is  no  contact  with  the  enemy.  Once  the  enemy  has  been  sighted  and  is  within  weapons 
range  this  phase  of  the  scenario  ends.  During  this  phase  the  activities  of  the  FSO  become  more 
diverse  according  to  the  CoHOST  simulation  with  workload  and  utilization  rates  constantly 
changing  as  the  FSO  works  to  meet  constantly  changing  requirements  for  possible  artillery 
support  as  the  forces  move  forward. 

•  Table  3 1  shows  those  tasks,  predicted  by  JNNS  at  the  -10%  PSF  level,  during  this 
tactical  phase  whose  increased  utilization  can  cause  these  task  time  reduction 
effects.  The  activities  that  can  contribute  to  an  overall  10%  reduction  in  task  time 
requirements  according  to  the  JNNS  simulation  include  the  presence  of  the 
executive  officer  (XO)  and  the  S3  non-commissioned  officer  (S3NCO)  in  the 
TOC,  the  coordination  and  employment  of  integrated  fire  support  and  active  use 
of  systems  that  improve  command  and  control  or  those  things  that  improve 
continuous  planning  and  directing  of  tactical  operations.  Systems  that  can  affect 
these  activities  include  the  AFATDS,  the  maneuver  control  system  (MCS)  which 
is  the  digital  system  used  to  communicate  with  friendly  forces,  and  the  effective 
employment  of  reconnaissance  operations. 

Table  31  -  DV  /  IV  Correlation  -  MTC  -  PSF  @  -10% 


Phase  2-  MTC  (DV  @  0.25,  -10%  PSF) 


Dependent  Variable  /  Model 

Independent  Variable 

Multi¬ 

plication 

Factor 

#  02  -  Primary  Taskl  1.  Fixing 
enemy  in  position. 

#03  -  Intel  BOS-  3  Process  Information 

1 

#  02  -  Primary  Taskl  1.  Fixing 
enemy  in  position. 

#57  -  Misc  Factors-  Staff  Fluddle 

1 

#  02  -  Primary  Taskl  1.  Fixing 
enemy  in  position. 

#63  -  Personnel-  S3  RTO  present  in  TOC 

2 

#  07  -  Secondary  Taskl2. 

Synchronization  with 

#05  -  Maneuver  BOS-  1  Conduct  Tactical 
Movement 

2 
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Dependent  Variable  /  Model 

Independent  Variable 

Multi¬ 

plication 

Factor 

supporting  forces. 

#  07  -  Secondary  Taskl2. 

Synchronization  with 
supporting  forces. 

#15  -  Fire  Support  BOS-  9  Coordinate, 
Synchronize,  and  Integrate  F  ire 
Support 

2 

#  07  -  Secondary  Taskl2. 

Synchronization  with 
supporting  forces. 

#16  -  Mobility  &  Survivability  BOS-  1 
Overcome  Obstacles 

2 

#  08  -  Secondary  Taskl8. 

Deception  activities. 

#25  -  Command  and  Control  BOS-  1  Plan 
for  Combat  Operations 

2 

#  15  -  Mission2.  Movement  to 
Contact 

#08  -  Fire  Support  BOS-  2  Employ  Field 
Artillery 

1 

#  15  -  Mission2.  Movement  to 
Contact 

#27  -  Command  and  Control  BOS-  3  Direct 
and  Lead  Unit  in  Execution  of 

Battle 

1 

#  15  -  Mission2.  Movement  to 
Contact 

#62  -  Personnel-  S3  NCO  present  in  TOC 

1 

•  Table  32  shows  those  tasks,  predicted  by  JNNS  at  the  -20%  PSF  level,  during  this 
tactical  phase  whose  increased  utilization  can  cause  these  task  time  reduction 
effects.  Decreasing  the  task  perfonnance  time  during  this  MTC  phase  to  the  20% 
level  below  default  is  predicted  as  being  possible  by  the  same  activities  that 
predict  a  10%  reduction  but  with  utilization  rates. 

Table  32  -  DV  /  IV  Correlation  -  MTC  -  PSF  @  -20% 


Phase  2-  MTC  (DV  @  0.5,  -20%  PSF) 


Dependent  Variable  /  Model 

Independent  Variable 

Multi¬ 

plication 

Factor 

#  02  -  Primary  Taskl  1.  Fixing 
enemy  in  position. 

#03  -  Intel  BOS-  3  Process  Information 

1 

#  02  -  Primary  Taskl  1.  Fixing 
enemy  in  position. 

#26  -  Command  and  Control  BOS-  2  Direct 
and  Lead  Unit  During  the 

Preparation  Phase  of  the  Battle 

2 

#  02  -  Primary  Taskl  1.  Fixing 
enemy  in  position. 

#57  -  Misc  Factors-  Staff  Huddle 

1 

#  02  -  Primary  Taskl  1.  Fixing 
enemy  in  position. 

#59  -  Personnel-  Executive  Officer  present 
in  TOC 

1 

#  07  -  Secondary  Taskl2. 

Synchronization  with 
supporting  forces. 

#05  -  Maneuver  BOS-  1  Conduct  Tactical 
Movement 

2 

#  07  -  Secondary  Taskl2. 

Synchronization  with 
supporting  forces. 

#15  -  Fire  Support  BOS-  9  Coordinate, 
Synchronize,  and  Integrate  F  ire 
Support 

1 

#  07  -  Secondary  Taskl2. 

Synchronization  with 

#16  -  Mobility  &  Survivability  BOS-  1 
Overcome  Obstacles 

1 
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Dependent  Variable  /  Model 

Independent  Variable 

Multi¬ 

plication 

Factor 

supporting  forces. 

#  08  -  Secondary  Taskl8. 

Deception  activities. 

#08  -  Fire  Support  BOS-  2  Employ  Field 
Artillery 

2 

#  08  -  Secondary  Taskl8. 

Deception  activities. 

#25  -  Command  and  Control  BOS-  1  Plan 
for  Combat  Operations 

1 

#  15  -  Mission2.  Movement  to 
Contact 

#27  -  Command  and  Control  BOS-  3  Direct 
and  Lead  Unit  in  Execution  of 

Battle 

1 

#  15  -  Mission2.  Movement  to 
Contact 

#52  -  Misc  Factors-  Reconnaissance 
Operations 

1 

#  15  -  Mission2.  Movement  to 
Contact 

#62  -  Personnel-  S3  NCO  present  in  TOC 

1 

•  Table  33  shows  those  tasks,  predicted  by  JNNS  at  the  -30%  PSF  level,  during  this 
tactical  phase  whose  increased  utilization  can  cause  these  task  time  reduction 
effects.  Reducing  the  task  performance  times  30%  below  default  is  predicted  as 
being  possible  through  a  continuation  of  existing  activities  along  with  higher 
activities  with  fire  support  battlefield  operating  systems  (BOS)  and  more  effective 
mobility  operations. 

Table  33  -  DV  /  IV  Correlation  -  MTC  -  PSF  @  -30% 


Phase  2-  MTC  (DV  @  0.75,  -30%  PSF) 


Dependent  Variable  /  Model 

Independent  Variable 

Multi¬ 

plication 

Factor 

#01  -  Secondary  Taskl2. 

Synchronization  with 
supporting  forces. 

#05  -  Maneuver  BOS-  1  Conduct  Tactical 
Movement 

2 

#  07  -  Secondary  Taskl2. 

Synchronization  with 
supporting  forces. 

#15  -  Fire  Support  BOS-  9  Coordinate, 
Synchronize,  and  Integrate  F  ire 
Support 

2 

#  07  -  Secondary  Taskl2. 

Synchronization  with 
supporting  forces. 

#16  -  Mobility  &  Survivability  BOS-  1 
Overcome  Obstacles 

3 

#  08  -  Secondary  Taskl8. 

Deception  activities. 

#08  -  Fire  Support  BOS-  2  Employ  Field 
Artillery 

3 

#  08  -  Secondary  Taskl8. 

Deception  activities. 

#21  -  Mobility  &  Survivability  BOS-  6 
Conduct  Deception 

1 

#  08  -  Secondary  Taskl8. 

Deception  activities. 

#59  -  Personnel-  Executive  Officer  present 
in  TOC 

1 

#11-  Tertiary  Task3. 

Movement  to  the  line 
of  departure. 

#25  -  Command  and  Control  BOS-  1  Plan 
for  Combat  Operations 

1 

#11-  Tertiary  Task3. 

Movement  to  the  line 

#63  -  Personnel-  S3  RTO  present  in  TOC 

1 
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Dependent  Variable  /  Model 

Independent  Variable 

Multi¬ 

plication 

Factor 

of  departure. 

#  15  -  Mission2.  Movement  to 
Contact 

#26  -  Command  and  Control  BOS-  2  Direct 
and  Lead  Unit  During  the 

Preparation  Phase  of  the  Battle 

1 

#  15  -  Mission2.  Movement  to 
Contact 

#27  -  Command  and  Control  BOS-  3  Direct 
and  Lead  Unit  in  Execution  of 

Battle 

1 

#  15  -  Mission2.  Movement  to 
Contact 

#52  -  Misc  Factors-  Reconnaissance 
Operations 

1 

#  15  -  Mission2.  Movement  to 
Contact 

#62  -  Personnel-  S3  NCO  present  in  TOC 

1 

4.2. 2. 3.  Scenario  Phase  3  -  Attack. 

This  phase  of  the  scenario  is  characterized  by  the  fact  that  the  friendly  forces  have 
encountered  the  enemy  and  are  attacking  them.  The  enemy  may  also  be  moving  thus  causing  a 
meeting  engagement,  or  they  may  be  in  a  stationary  defense  where  the  action  becomes  an  attack 
against  fixed  fortifications.  In  this  situation  the  FSO  is  fairly  constantly  utilized  with  a  workload 
that  includes  the  actual  directing  and  coordinating  of  artillery  and  airstrike  missions  against  the 
enemy. 

•  Table  34  shows  those  tasks,  predicted  by  JNNS  at  the  -10%  PSF  level,  during  this 
tactical  phase  whose  increased  utilization  can  cause  these  task  time  reduction 
effects.  10%  reductions  in  task  time  can  be  influenced  by  the  presence  of  the  XO, 
negatively  by  TOC  personnel  being  in  chemical  protective  clothing  (MOPP)  and 
negatively  by  abnormally  high  or  low  humidity.  The  key  activity,  however,  is  the 
actual  employment  of  field  artillery  assets  as  indicated  by  a  higher  multiplication 
factor  in  the  table.  For  the  FSO,  the  key  to  success  in  this  effort  is  effective  use  of 
communications  systems  both  to  know  and  understand  the  enemy  situation  as 
well  as  the  ability  to  direct  the  artillery  assets  supporting  the  battle.  The 
AFATDS  and  ASAS  digital  communications  systems  provide  this 
communications  abilities  and  an  increased  efficiency  to  this  perfonnance  level 
could  be  the  key  to  these  improvement  effects. 
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Table  34  -  DV  /  IV  Correlation  -  Attack  -  PSF  @  -10% 


Phase  3-  Attack  (DV  @  0.25,  -10%  PSF) 


Dependent  Variable  /  Model 

Independent  Variable 

Multi¬ 

plication 

Factor 

#  03  -  Primary  Taskl2. 

Synchronization  with 
supporting  forces. 

#08  -  Fire  Support  BOS-  2  Employ  Field 
Artillery 

2 

#  03  -  Primary  Taskl2. 

Synchronization  with 
supporting  forces. 

#21  -  Mobility  &  Survivability  BOS-  6 
Conduct  Deception 

1 

#  03  -  Primary  Taskl2. 

Synchronization  with 
supporting  forces. 

#25  -  Command  and  Control  BOS-  1  Plan 
for  Combat  Operations 

1 

#  03  -  Primary  Taskl2. 

Synchronization  with 
supporting  forces. 

#69  -  Environmental  Factor-  Relative 
Flumidity,  % 

1 

#  04  -  Primary  Taskl5. 

Destruction  of  first 
echelon  forces. 

#50  -  Misc  Factors-  Battle  Timing 

1 

#  04  -  Primary  Taskl5. 

Destruction  of  first 
echelon  forces. 

#59  -  Personnel-  Executive  Officer  present 
in  TOC 

1 

#  16  -  Mission3.  Attack 

#56  -  Misc  Factors-  MOPP  Level 

1 

#  17  -  Mission5.  River 

Crossing 

#24  -  Air  Defense  BOS-  2  Take  Passive  Air 
Defense  Measures 

1 

•  Table  35  shows  those  tasks,  predicted  by  JNNS  at  the  -20%  PSF  level,  during  this 
tactical  phase  whose  increased  utilization  can  cause  these  task  time  reduction 
effects.  The  measures  that  are  predicted  to  cause  additional  efficiencies  to  this 
level  include  the  intelligence  BOS  of  intelligence  dissemination,  and  battle  timing 
or  the  ability  to  synchronize  forces  on  the  battlefield  to  meet  enemy  threats. 
Providing  a  strong  negative  correlation  is  the  performance  degradation  while  in 
MOPP  status  (when  the  TOC  personnel  are  wearing  chemical  protective  clothing 
and  equipment).  The  presence  of  the  XO  and  S3NCO  in  the  TOC  is  also 
predicted  to  contribute  to  this  increased  perfonnance  level. 
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Table  35  -  DV  /  IV  Correlation  -  Attack  -  PSF  @  -20% 


Phase  3-  Attack  (DV  @  0.5,  -20%  PSF) 


Dependent  Variable  /  Model 

Independent  Variable 

Multi¬ 

plication 

Factor 

#  01  -  Primary  TasklO. 

Destruction,  capture, 
or  bypass  of  enemy 
force. 

#04  -  Intel  BOS-  4  Disseminate 

Intelligence 

4 

#  01  -  Primary  TasklO. 

Destruction,  capture, 
or  bypass  of  enemy 
force. 

#17  -  Mobility  &  Survivability  BOS-  2 
Enhance  Movement 

1 

#  01  -  Primary  TasklO. 

Destruction,  capture, 
or  bypass  of  enemy 
force. 

#26  -  Command  and  Control  BOS-  2  Direct 
and  Lead  Unit  During  the 

Preparation  Phase  of  the  Battle 

1 

#  01  -  Primary  TasklO. 

Destruction,  capture, 
or  bypass  of  enemy 
force. 

#56  -  Misc  Factors-  MOPP  Level 

3 

#  03  -  Primary  Taskl2. 

Synchronization  with 
supporting  forces. 

#21  -  Mobility  &  Survivability  BOS-  6 
Conduct  Deception 

1 

#  03  -  Primary  Taskl2. 

Synchronization  with 
supporting  forces. 

#22  -  Mobility  &  Survivability  BOS-  7 
Conduct  NBC  Defense 

1 

#  03  -  Primary  Taskl2. 

Synchronization  with 
supporting  forces. 

#50  -  Misc  Factors-  Battle  Timing 

3 

#  04  -  Primary  Taskl5. 

Destruction  of  first 
echelon  forces. 

#59  -  Personnel-  Executive  Officer  present 
in  TOC 

1 

#  17  -  Mission5.  River 

Crossing 

#62  -  Personnel-  S3  NCO  present  in  TOC 

1 

•  Table  36  shows  those  tasks,  predicted  by  JNNS  at  the  -30%  PSF  level,  during  this 
tactical  phase  whose  increased  utilization  can  cause  these  task  time  reduction 
effects.  Adding  to  many  of  the  same  factors  present  at  the  -20%  PSF  level  is 
success  at  the  employment  of  field  artillery  assets  and  the  effective  use  of  the  field 
artillery  digital  communications  system  AFATDS. 
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Table  36  -  DV  /  IV  Correlation  -  Attack  -  PSF  @  -30% 


Phase  3-  Attack  (DV  @  0.75,  -30%  PSF) 


Dependent  Variable  /  Model 

Independent  Variable 

Multi¬ 

plication 

Factor 

#  01  -  Primary  TasklO. 

Destruction,  capture, 
or  bypass  of  enemy 
force. 

#04  -  Intel  BOS-  4  Disseminate 

Intelligence 

3 

#  01  -  Primary  TasklO. 

Destruction,  capture, 
or  bypass  of  enemy 
force. 

#17  -  Mobility  &  Survivability  BOS-  2 
Enhance  Movement 

1 

#  01  -  Primary  TasklO. 

Destruction,  capture, 
or  bypass  of  enemy 
force. 

#26  -  Command  and  Control  BOS-  2  Direct 
and  Lead  Unit  During  the 

Preparation  Phase  of  the  Battle 

1 

#  01  -  Primary  TasklO. 

Destruction,  capture, 
or  bypass  of  enemy 
force. 

#56  -  Misc  Factors-  MOPP  Level 

2 

#  03  -  Primary  Taskl2. 

Synchronization  with 
supporting  forces. 

#08  -  Fire  Support  BOS-  2  Employ  Field 
Artillery 

2 

#  03  -  Primary  Taskl2. 

Synchronization  with 
supporting  forces. 

#15  -  Fire  Support  BOS-  9  Coordinate, 
Synchronize,  and  Integrate  F  ire 
Support 

1 

#  03  -  Primary  Taskl2. 

Synchronization  with 
supporting  forces. 

#35  -  Communications-  AFATDS  - 

Advanced  Field  Artillery  Tactical 
Data  System 

1 

#  03  -  Primary  Taskl2. 

Synchronization  with 
supporting  forces. 

#67  -  Personnel-  ALO  present  in  TOC 

1 

#  03  -  Primary  Taskl2. 

Synchronization  with 
supporting  forces. 

#69  -  Environmental  Factor-  Relative 
Flumidity,  % 

1 

#  04  -  Primary  Taskl5. 

Destruction  of  first 
echelon  forces. 

#50  -  Misc  Factors-  Battle  Timing 

2 

#  04  -  Primary  Taskl5. 

Destruction  of  first 
echelon  forces. 

#59  -  Personnel-  Executive  Officer  present 
in  TOC 

1 

#  17  -  Mission5.  River 

Crossing 

#05  -  Maneuver  BOS-  1  Conduct  Tactical 
Movement 

1 

#  17  -  Mission5.  River 

Crossing 

#24  -  Air  Defense  BOS-  2  Take  Passive  Air 
Defense  Measures 

1 

#  17  -  Mission5.  River 

Crossing 

#62  -  Personnel-  S3  NCO  present  in  TOC 

1 
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4.2. 2.4.  Scenario  Phase  4  -  Defense 


This  phase  of  the  scenario  is  characterized  by  the  fact  that  the  friendly  forces  have  seized 
or  otherwise  occupied  a  battle  position  and  they  conduct  a  defense  against  enemy  forces  who  try 
to  take  the  position  away  from  them  through  hostile  attack.  In  this  situation,  as  in  the  attack 
phase  of  the  scenario,  the  FSO  is  fairly  constantly  utilized  with  a  workload  that  includes  the 
actual  directing  and  coordinating  of  artillery  and  airstrike  missions  against  the  enemy. 


•  Table  37  shows  those  tasks,  predicted  by  JNNS  at  the  -10%  PSF  level,  during  this 
tactical  phase  whose  increased  utilization  can  cause  these  task  time  reduction 
effects.  10%  reductions  are  predicted  to  be  achieved  most  importantly  through 
effective  employment  of  field  artillery  assets.  Also  contributing  are  the  effective 
coordination  of  field  artillery  assets,  an  effective  plan  for  the  conduct  of  the 
defense,  and  good  battle  rhythm  or  battle  timing  in  the  TOC  operation.  The 
presence  of  the  XO  is  also  predicted  to  contribute  to  TOC  effectiveness 
improvements  to  this  level. 

Table  37  -  DV  /  IV  Correlation  -  Defense  -  PSF  @  -10% 


Phase  4-  Defense  (DV  @  0.25,  -10%  PSF) 


Dependent  Variable  /  Model 

Independent  Variable 

Multi¬ 

plication 

Factor 

#  03  -  Primary  Taskl2. 

Synchronization  with 
supporting  forces. 

#08  -  Fire  Support  BOS-  2  Employ  Field 
Artillery 

2 

#  03  -  Primary  Taskl2. 

Synchronization  with 
supporting  forces. 

#21  -  Mobility  &  Survivability  BOS-  6 
Conduct  Deception 

1 

#  03  -  Primary  Taskl2. 

Synchronization  with 
supporting  forces. 

#25  -  Command  and  Control  BOS-  1  Plan 
for  Combat  Operations 

1 

#  03  -  Primary  Taskl2. 

Synchronization  with 
supporting  forces. 

#69  -  Environmental  Factor-  Relative 
Flumidity,  % 

1 

#  04  -  Primary  Taskl5. 

Destruction  of  first 
echelon  forces. 

#50  -  Misc  Factors-  Battle  Timing 

1 

#  04  -  Primary  Taskl5. 

Destruction  of  first 
echelon  forces. 

#59  -  Personnel-  Executive  Officer  present 
in  TOC 

1 

#  07  -  Secondary  Taskl2. 

Synchronization  with 
supporting  forces. 

#05  -  Maneuver  BOS-  1  Conduct  Tactical 
Movement 

1 
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Dependent  Variable  /  Model 

Independent  Variable 

Multi¬ 

plication 

Factor 

#  07  -  Secondary  Taskl2. 

Synchronization  with 
supporting  forces. 

#15  -  Fire  Support  BOS-  9  Coordinate, 
Synchronize,  and  Integrate  F  ire 
Support 

1 

#  07  -  Secondary  Taskl2. 

Synchronization  with 
supporting  forces. 

#16  -  Mobility  &  Survivability  BOS-  1 
Overcome  Obstacles 

1 

•  Table  38  shows  those  tasks,  predicted  by  JNNS  at  the  -20%  PSF  level,  during  this 
tactical  phase  whose  increased  utilization  can  cause  these  task  time  reduction 
effects.  In  addition  to  factors  at  the  10%  level  such  as  battle  timing  and  the 
presence  of  the  XO,  20%  reductions  are  predicted  to  be  achieved  through  the 
effective  dissemination  of  intelligence  and  effective  counter  battery  target 
operations  where  enemy  artillery  is  identified,  located,  and  fired  upon  with 
friendly  artillery  and  airstrikes.  Effective  use  of  the  ASAS  and  AFATDS  digital 
communications  systems  along  with  effective  liaison  with  the  ALO  can  contribute 
to  these  effects. 

Table  38  -  DV  /  IV  Correlation  -  Defense  -  PSF  @  -20% 


Phase  4-  Defense  (DV  @  0.5,  -20%  PSF) 


Dependent  Variable  /  Model 

Independent  Variable 

Multi¬ 

plication 

Factor 

#  01  -  Primary  TasklO. 

Destruction,  capture, 
or  bypass  of  enemy 
force. 

#04  -  Intel  BOS-  4  Disseminate 

Intelligence 

2 

#  01  -  Primary  TasklO. 

Destruction,  capture, 
or  bypass  of  enemy 
force. 

#17  -  Mobility  &  Survivability  BOS-  2 
Enhance  Movement 

1 

#  01  -  Primary  TasklO. 

Destruction,  capture, 
or  bypass  of  enemy 
force. 

#26  -  Command  and  Control  BOS-  2  Direct 
and  Lead  Unit  During  the 

Preparation  Phase  of  the  Battle 

1 

#  01  -  Primary  TasklO. 

Destruction,  capture, 
or  bypass  of  enemy 
force. 

#56  -  Misc  Factors-  MOPP  Level 

1 

#  03  -  Primary  Taskl2. 

Synchronization  with 
supporting  forces. 

#13  -  Fire  Support  BOS-  7  Conduct 

Counter  Target  Acquisition 
Operations 

1 

#  03  -  Primary  Taskl2. 

#21  -  Mobility  &  Survivability  BOS-  6 

1 
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Dependent  Variable  /  Model 

Independent  Variable 

Multi¬ 

plication 

Factor 

Synchronization  with 
supporting  forces. 

Conduct  Deception 

#  03  -  Primary  Taskl2. 

Synchronization  with 
supporting  forces. 

#50  -  Misc  Factors-  Battle  Timing 

2 

#  04  -  Primary  Taskl5. 

Destruction  of  first 
echelon  forces. 

#59  -  Personnel-  Executive  Officer  present 
in  TOC 

1 

#  07  -  Secondary  Taskl2. 

Synchronization  with 
supporting  forces. 

#05  -  Maneuver  BOS-  1  Conduct  Tactical 
Movement 

1 

#  07  -  Secondary  Taskl2. 

Synchronization  with 
supporting  forces. 

#15  -  Fire  Support  BOS-  9  Coordinate, 
Synchronize,  and  Integrate  F  ire 
Support 

1 

#  07  -  Secondary  Taskl2. 

Synchronization  with 
supporting  forces. 

#16  -  Mobility  &  Survivability  BOS-  1 
Overcome  Obstacles 

1 

•  Table  39  shows  those  tasks,  predicted  by  JNNS  at  the  -30%  PSF  level,  during  this 
tactical  phase  whose  increased  utilization  can  cause  these  task  time  reduction 
effects.  30%  reductions  are  predicted  to  be  achieved  through  most  of  the  same 
effects  as  existed  at  the  20%  level  with  increased  emphasis  on  the  use  of 
AFATDS  and  the  more  effective  use  of  the  field  artillery  BOSs. 

Table  39  -  DV  /  IV  Correlation  -  Defense  -  PSF  @  -30% 


Phase  4-  Defense  (DV  @  0.75,  -30%  PSF) 


Dependent  Variable  /  Model 

Independent  Variable 

Multi¬ 

plication 

Factor 

#  01  -  Primary  TasklO. 

Destruction,  capture, 
or  bypass  of  enemy 
force. 

#04  -  Intel  BOS-  4  Disseminate 

Intelligence 

2 

#  01  -  Primary  TasklO. 

Destruction,  capture, 
or  bypass  of  enemy 
force. 

#17  -  Mobility  &  Survivability  BOS-  2 
Enhance  Movement 

1 

#  01  -  Primary  TasklO. 

Destruction,  capture, 
or  bypass  of  enemy 
force. 

#26  -  Command  and  Control  BOS-  2  Direct 
and  Lead  Unit  During  the 

Preparation  Phase  of  the  Battle 

1 

#  01  -  Primary  TasklO. 

Destruction,  capture, 
or  bypass  of  enemy 
force. 

#56  -  Misc  Factors-  MOPP  Level 

1 
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Dependent  Variable  /  Model 

Independent  Variable 

Multi¬ 

plication 

Factor 

#  03  -  Primary  Taskl2. 

Synchronization  with 
supporting  forces. 

#08  -  Fire  Support  BOS-  2  Employ  Field 
Artillery 

2 

#  03  -  Primary  Taskl2. 

Synchronization  with 
supporting  forces. 

#15  -  Fire  Support  BOS-  9  Coordinate, 
Synchronize,  and  Integrate  F  ire 
Support 

2 

#  03  -  Primary  Taskl2. 

Synchronization  with 
supporting  forces. 

#35  -  Communications-  AFATDS  - 

Advanced  Field  Artillery  Tactical 
Data  System 

1 

#  03  -  Primary  Taskl2. 

Synchronization  with 
supporting  forces. 

#67  -  Personnel-  ALO  present  in  TOC 

1 

#  03  -  Primary  Taskl2. 

Synchronization  with 
supporting  forces. 

#69  -  Environmental  Factor-  Relative 
Humidity,  % 

1 

#  04  -  Primary  Taskl5. 

Destruction  of  first 
echelon  forces. 

#50  -  Misc  Factors-  Battle  Timing 

1 

#  04  -  Primary  Taskl5. 

Destruction  of  first 
echelon  forces. 

#59  -  Personnel-  Executive  Officer  present 
in  TOC 

1 

#  07  -  Secondary  Taskl2. 

Synchronization  with 
supporting  forces. 

#05  -  Maneuver  BOS-  1  Conduct  Tactical 
Movement 

1 

#  07  -  Secondary  Taskl2. 

Synchronization  with 
supporting  forces. 

#16  -  Mobility  &  Survivability  BOS-  1 
Overcome  Obstacles 

1 

4.2.3.  Summary  of  Results. 

The  analyses  presented  in  this  section  illustrate  the  successful  use  of  the  COMPASS 
framework.  A  systematic  step  by  step  approach  has  been  shown  that  allows  a  cross-walk 
identification  of  the  mission  perfonnance  effects  as  compared  to  human  performance 
characteristics.  These  impacts  on  human  perfonnance  capabilities  provide  the  means  to  identify 
C2S  subsystems  whose  performance  either  successfully  or  unsuccessfully  contributes  to  overall 
changes  in  the  way  the  TOC  functions.  An  unsuccessful  contribution  of  the  perfonnance  of  a 
particular  subsystem  is  considered  to  be  just  as  important  in  the  overall  analysis  as  a  successful 
one.  If  the  efficient  performance  of  a  C2S  subsystem,  such  as  an  individual  intelligence 
collection  asset,  actually  causes  an  overall  degradation  in  the  performance  capability  of  the  C2S, 
then  that  is  an  important  piece  of  information  for  C2S  designers.  The  conclusion  might  then  be 
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to  eliminate  or  alter  the  characteristics  of  that  subsystem  rather  than  continuing  to  try  and 
improve  it. 
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5.  Discussion. 


The  processes,  discussions,  and  analyses  in  Phase  I  and  Phase  II  have  provided  a  proof  of 
concept  that  an  approach  such  as  provided  by  the  COMPASS  paradigm  can  be  used  to  look  at 
complex  work  domains.  Certainly,  there  has  been  much  highly  creditable  work  performed  over 
the  last  50  years  in  the  area  of  military  C2  in  general  and  various  aspects  of  it  in  specific.  Topics 
such  as  decision  making  (including  MDMP),  individual  and  team  performance,  cognition,  task 
analysis,  knowledge  elicitation,  workload,  situation  awareness,  and  span  of  control  are  but  a  few 
of  the  highly  relevant  areas  of  investigation  that  have  been  explored  in  this  domain.  However, 
the  complexity  of  each  of  these  research  areas  and  the  open  ended  nature  of  the  research  in  them 
has  apparently  precluded  substantive  efforts  at  looking  at  the  overall  work  system  in  military  C2. 

Underlying  the  development  of  the  COMPASS  paradigm  is  an  acknowledgement  that  all 
of  these  areas  are  critical  and  highly  significant  in  their  own  right,  however,  an  understanding  of 
what  makes  the  overall  system  perfonn  more  or  less  effectively  is  the  key  to  understanding  how 
to  improve  it.  The  use  of  COMPASS  to  investigate  the  overall  C2S  as  a  system  with  its  own 
performance  parameters  is  the  goal  here.  Some  of  these  performance  parameters  are  human 
based,  some  are  hardware  and  software  based,  some  are  environmentally  based,  and  some  are 
based  upon  the  interaction  from  and  to  other  like  systems.  Implicit  in  this  definition  is  an 
acknowledgement  of  the  role  that  each  of  the  subsystem  areas  play  in  the  perfonnance  of  the 
system  while  attempting  to  account  for  the  effects  of  these  internal  systems  to  the  effectiveness 
of  the  overall  system. 

The  techniques  used  to  develop  the  COMPASS  paradigm  have  been  exploratory  in  nature 
and  represent  general  classes  of  approaches  that  are  appropriate  for  future  efforts.  The  field  of 
datamining  uses  many  approaches  such  as  cluster  analysis  and  decision  trees  in  addition  to  the 
neural  network  evaluations  performed  here.  These  areas,  and  others,  certainly  provide 
appropriate  approaches  for  future  development  and  improvement  of  the  COMPASS  approach. 

5.1.  The  COMPASS  Paradigm. 

The  COMPASS  paradigm  has  allowed  the  presentation  of  a  hypothesis  that  postulates 
that  decreasing  the  amount  of  time  required  to  perfonn  individual  tasks  in  a  TOC  based  work 
environment  will  result  in  more  effective  operator  performance  up  to  the  point  of  operator 
overload.  This  paradigm  has  the  potential  to  investigate  many  other  issues  in  this  work  domain. 
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Future  research  will  cover  other  performance  areas.  Some  of  these  performance  areas  could 
include  changes  in  the  decision  maker’s  battlefield  situation  awareness  through  increased  volume 
of  communication  traffic  or  through  better  structured  communications  traffic.  Analyses  that  look 
at  the  significance  of  the  contribution  of  the  activities  of  individual  operators  to  overall  mission 
success  could  be  performed.  Finally,  the  ability  to  provide  the  medium  for  the  investigation  for, 
as  yet,  undefined  performance  areas  from  a  pre-existing  and  growing  knowledge  base  is  perhaps 
the  biggest  potential  contribution  of  the  COMPASS  paradigm.  All  of  this  current  and  future 
research  directly  contributes  to  the  U.S.  Army’s  efforts  to  improve  the  integration  of  the  human 
system  component  into  overall  socio-technical  work  system  designs.  Ongoing  initiatives  that 
can  benefit  from  research  of  this  type,  such  as  the  Army  MANPRINT  program,  are  critical  to  the 
effectiveness  of  tomorrow’s  C2  systems.  These  benefits  can  only  be  achieved  if  new  and 
innovative  research  programs  are  pursued  that  can  address  the  complexities  of  the  work  systems 
involved. 

The  COMPASS  paradigm  has  proved  to  be  a  viable  framework  to  examine  TOC 
performance  at  the  system  level.  The  demands  that  war  places  on  the  complex  battlefield  of 
today  means  that  the  systems  and  subsystems  designed  to  assist  human  operators  in  their 
attempts  to  control  that  battlefield  must  effectively  allow  an  improved  performance  of  the 
operator  and  subsequently  the  overall  system.  This  is  a  complicated  performance  arena  where 
the  effects  of  new  automation,  communications,  and  organizational  structures  cannot  be  readily 
ascertained.  One  trend  of  modem  C2  is  to  employ  new  automation  to  enable  smaller  groups  of 
individuals  to  exercise  direct  control  over  greatly  expanded  forces  and  terrain.  Previous  studies 
with  the  CoHOST  model  have  mirrored  subsequent  field  observations  that  show  that  just 
providing  a  more  efficient  communications  capability  can  force  a  data  overload  state  on  the  TOC 
operators  and  decision  makers  with  less,  not  more,  efficiency  in  their  decision  making  abilities. 

It  is  in  response  to  this  complicated  performance  environment  that  has  directed  the 
development  of  the  COMPASS  approach.  The  evaluation  of  human  cognitive  performance  is  a 
challenge  in  any  work  environment,  but  military  C2  TOCs  perform  at  a  level  of  sophistication 
that  is  hard  to  detect  by  the  initiated  observer  and  processes,  but  far  surpasses  any  conceivable 
peacetime  effort.  The  heart  of  the  COMPASS  approach  is  that  it  is  not  a  theoretical  estimation 
of  TOC  performance,  but  is  rooted  in  direct  observation  of  actual  performing  systems  over  time 
to  build  a  knowledge  base  of  what  is  and  is  not  significant  within  the  TOC  work  system.  The 
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key  to  this  approach  is  the  establishment  of  the  ability  to  take  the  results  of  factor  screening 
methods,  such  as  this  use  of  the  JNNS,  and  apply  the  resultant  significant  factors  to  simulations 
of  human  system  perfonnance  such  as  the  CoHOST  TOC  simulation. 

The  significance  of  this  research  is  the  development  of  the  COMPASS  paradigm.  This  is 
a  two  phase  procedure  that  consists  of  naturalistic  observation  of  multiple  TOC  perfonnance 
over  time  along  with  a  neural  network  evaluation  of  the  data  and  simulation  studies  of  the 
performance  of  the  TOC  using  a  discrete  event  computer  simulation  that  utilizes  a  task  network 
modeling  approach  whose  performance  was  shaped  by  the  models  from  the  neural  network 
simulation.  This  paradigm  is  illustrated  in  Figure  39. 


Figure  39  -  Block  Diagram  of  the  COMPASS  Paradigm 
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The  naturalistic  observation  and  data  gathering  performed  here  looked  at  three  TOC 
deployments.  These  data  provided  a  knowledge  set  sufficient  to  develop  and  test  COMPASS 
and  to  provide  preliminary  results  that  describe  some  of  the  factors  that  significantly  affect  C2 
performance  in  TOCs.  Following  this  dissertation  effort,  over  the  next  several  years,  it  is 
envisioned  that  this  research  will  be  extended  to  the  observation  of  many  future  TOC 
deployments  and  the  generation  of  a  knowledge  base  from  the  observation  of  40  or  more  such 
events.  As  this  knowledge  set  grows,  so  will  the  ability  to  investigate  performance  relationships 
among  the  combined  data  that  will  describe  those  significant  elements  that  detennine  efficient 
TOC  operation  while  transcending  the  interpersonal  differences  in  the  areas  of  task  and  team 
performance  that  exist  within  each  individual  TOC  exercise. 

A  series  of  guidelines,  derived  from  these  methods,  is  presented  to  formalize  the 
COMPASS  concept.  These  guidelines  are  identified  in  Table  40.  It  is  noted  that  these 
guidelines  are  for  the  JNNS  and  CoHOST  simulations.  The  process  may  be  altered  slightly  if 
other  neural  network  and  discrete  event  simulations  are  employed. 


Table  40  -  Guidelines  For  The  Use  Of  The  COMPASS  Paradigm 


Compass  Phase 

# 

Description 

Naturalistic  Observation 

1 

Visit  TOC  before  start  of  the  exercise  to  become  familiar  with  its  layout. 

Naturalistic  Observation 

2 

Obtain  copy  of  OPORD  before  start  of  the  exercise  to  identify  scenario(s). 

Naturalistic  Observation 

3 

Spend  the  first  few  hours  after  start  of  the  exercise  just  watching  to  ascertain 
flow  of  activity  in  the  TOC. 

Naturalistic  Observation 

4 

After  start  of  the  exercise  do  not  initiate  communication  or  try  to  elicit 
information  from  personnel  in  TOC  engaged  in  the  exercise. 

Naturalistic  Observation 

5 

Readily  and  immediately  respond  to  communication  attempts  initiated  by  TOC 
personnel.  After  answering  their  query,  use  the  opportunity  to  objectively 
discuss  the  exercise  and  to  query  them  about  what  they  think  about  what  is 
happening. 

Naturalistic  Observation 

6 

Treat  all  personnel  with  equal  respect  and  enthusiasm  regardless  of  rank  or  team 
assignment. 

Naturalistic  Observation 

7 

Regardless  of  personal  opinion,  always  display  a  positive  opinion  about  what  is 
happening,  then  make  objective  observations  about  the  real  situation.  Do  not 
share  raw  data  from  observations  with  TOC  personnel,  however,  be  frank  and 
honest  about  it  if  asked.  Never  lie  or  misrepresent  any  aspect  of  the  project,  or 
the  data. 

Naturalistic  Observation 

8 

Capitalize  on  personal  military  background  or  other  experience  to 
ethnographically  relate  to  TOC  personnel  to  gain  temporary  acceptance  to  the 
work  group  environment. 

Naturalistic  Observation 

9 

Longitudinally  record  observations  according  to  a  set  time  interval,  such  as 
hourly,  and  focus  on  consistency  and  repeatability  on  type  and  kind  of 
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Compass  Phase 

# 

Description 

observations. 

Naturalistic  Observation 

10 

Record  all  observations  quantitatively  using  direct  automated  input,  if  possible, 
to  preclude  possible  transcription  and  interpretation  errors  after  the  end  of  the 
exercise. 

Neural  Network  Simulation 

1 

Convert  all  independent  and  dependent  measures  to  either  binary  or 
standardized  (0,1)  IV  and  DV  as  appropriate. 

Neural  Network  Simulation 

2 

Reorganize  IV  and  DV  into  input  and  output  patterns  of  data  where  each  pattern 
is  one  set  of  observations. 

Neural  Network  Simulation 

3 

Randomly  select  15%  -  25%  of  the  pattern  data  and  separate  it  into  a  validation 
pattern  set.  Reclassify  the  remaining  pattern  data  as  the  training  pattern  data 
set. 

Neural  Network  Simulation 

4 

Configure  the  input  layer  of  the  neural  network  with  one  node  per  IV. 

Neural  Network  Simulation 

5 

Configure  the  hidden  layer(s)  of  the  neural  network  with  the  minimum  number 
of  layers  and  nodes  in  each  layer  that  will  support  successful  training.  This  may 
require  trial  and  error  test  runs  to  determine  the  optimum  configuration.  Start 
with  one  hidden  layer  and  a  number  of  nodes  equal  to  approximately  14  of  the 
number  of  input  nodes. 

Neural  Network  Simulation 

6 

Configure  the  output  layer  of  the  neural  network  with  one  node  per  DV. 

Neural  Network  Simulation 

7 

Set  the  training  threshold  parameters  and  run  the  neural  network  simulation  for 
a  fixed  number  of  epochs  (start  with  100  and  increase  until  full  training  is 
achieved).  Observe  the  error  curve  from  the  training  and  validation  data  sets. 
When  the  validation  curve  turns  up  the  point  of  full  training  is  indicated.  Do 
not  overtrain.  If  the  curves  flatten  out  and  do  not  turn  up,  the  network  is  stable 
and  will  not  overtrain. 

Neural  Network  Simulation 

8 

If  full  training  is  not  indicated,  increase  the  number  of  epochs  and  rerun  until  it 
is. 

Neural  Network  Simulation 

9 

Once  full  training  is  indicated,  set  the  pinning  parameters  and  perform  pinning 
of  the  IV  nodes.  The  nodes  removed  by  pruning  are  predicted  to  have  less 
significance  to  the  development  of  the  output  than  those  nodes  remaining. 

Neural  Network  Simulation 

10 

By  examination  of  the  network  diagram  and  comparison  of  the  pre  and  post 
pruning  network  results  files,  identify  those  input  nodes  that  are  remaining  as 
candidates  for  development  of  mathematical  models  of  their  response. 

Neural  Network  Simulation 

11 

Examine  the  training  dataset  and  identify  those  output  parameters  that  have  a 
sufficient  number  of  observations  to  support  development  of  models.  A 
subjective  number  of  10  was  chosen  as  the  minimum  number  of  observations 
that  would  support  model  generation. 

Neural  Network  Simulation 

12 

Using  traditional  linear  regression  analysis  procedures,  perform  a  stepwise 
linear  regression  analysis  of  each  DV  model.  Begin  each  DV  model  with  all  of 
the  IV  remaining  after  pruning.  Eliminate  IV  from  each  model  that  are  not 
significant  after  each  regression  run  (a  =  .05)  and  rerun  the  regression  analysis. 
Continue  repeating  the  regression  runs  with  stepwise  elimination  until  each 
model  only  contains  significant  IV. 

Neural  Network  Simulation 

13 

Identify  the  significant  DV  models  for  use  in  the  interface  to  the  discrete  event 
simulation. 

Discrete  Event  Simulation 

1 

For  each  DV  model,  determine  the  range  of  its  response  by  setting  all  regressors 
to  first  0,  and  then  1 .  Record  the  results. 

Discrete  Event  Simulation 

2 

Identify  response  points  in  the  range  for  correlation  to  performance  points  in  the 
discrete  event  simulation  (for  example,  25%,  50%,  and  75%. ) 

Discrete  Event  Simulation 

3 

For  each  DV  model,  iteratively  adjust  the  values  of  the  regressors  (0  or  1  for 
binary  IV,  and  (0,1)  for  standardized  IV)  until  a  set  of  regressors  is  identified 
for  each  DV  model  that  contributes  to  the  0%,  25%,  50%,  75%,  and  100% 
response  of  the  model. 

Discrete  Event  Simulation 

4 

Using  SME  input,  identify  discrete  event  simulation  performance  shaping 
factors  (PSF)  that  correlate  to  response  points  of  the  DV  models.  For  example. 
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Compass  Phase 

# 

Description 

identify  a  PSF  that  causes  an  overall  reduction  in  task  time  performance  by  a 
fixed  percentile.  Then,  identify  a  range  of  these  PSFs  that  correlate  to  observed 
IV  utilization  in  the  DV  models.  The  values  used  in  this  dissertation  are: 

The  75%  DV  level  equates  to  a  PSF  =  approximately  -30%. 

The  50%  DV  level  equates  to  a  PSF  =  approximately  -20%. 

The  25%  DV  level  equates  to  a  PSF  =  approximately  -10%. 

The  0%  DV  level  equates  to  a  PSF  ==  0%  (default). 

For  example,  a  75%  increase  in  performance  of  a  DV  model  causes  a  30% 
decrease  in  task  time  performance  in  the  discrete  event  simulation. 

Discrete  Event  Simulation 

5 

Determine  the  minimum  number  of  replications,  R,  required  to  run  the  discrete 
event  simulation  to  account  for  the  effects  of  random  variability  using  Banks’ 
procedures. 

Discrete  Event  Simulation 

6 

For  each  phase  of  the  scenario,  use  SME  input  to  identify  those  DV  models  that 
are  applicable  to  the  performance  of  that  scenario  phase.  For  example  in  this 
dissertation,  during  the  scenario  phase  Movement  to  Contact,  it  was  determined 
that  the  performance  model  for  DV  TA3  (Movement  To  The  Line  Of 

Departure)  applied  to  this  combat  activity. 

Discrete  Event  Simulation 

7 

For  each  DV  model  associated  with  a  combat  scenario  phase,  list  its  IV  as 
factors  whose  utilization  contributes  to  the  performance  of  that  DV  during  the 
combat  phase.  For  all  the  DV  that  apply  to  each  combat  phase,  sum  all  of  the 
applying  IV  to  compile  a  list  of  IV  whose  level  of  utilization  contributes  to  the 
performance  of  operators  during  that  combat  phase. 

Discrete  Event  Simulation 

8 

Run  the  discrete  event  simulation  for  the  requisite  number  of  runs.  A  run  is 
defined  as  the  simulation  executing  the  full  scenario  for  a  PSF  factor  for  R 
replications.  Make  a  separate  run  for  each  PSF  factor. 

Discrete  Event  Simulation 

9 

Average  the  results  for  all  the  replications  in  each  run. 

Discrete  Event  Simulation 

10 

For  operators  of  interest,  separate  out  the  results  data  for  that  operator  and 
display  it  in  a  suitable  format  to  allow  evaluation  of  workload/taskload  and 
utilization  profiles,  and  numbers  of  tasks  that  were  queued,  dropped,  and 
interrupted  by  that  operator  during  the  run. 

Discrete  Event  Simulation 

11 

Evaluate  the  operator  performance  during  each  combat  phase  according  to 
taskload,  utilization,  and  task  performance.  Compare  the  operator  performance 
characteristics  during  each  combat  phase  to  the  IV  identified  in  step  7.  Derive 
conclusions  on  how  operator  performance  can  be  improved  to  this  level  through 
actions  that  can  be  taken  to  utilize  the  IV  to  that  performance  level. 

5.2.  Considerations  Regarding  The  COMPASS  Paradigm. 

It  can  be  generally  stated  that,  in  any  exploratory  research,  there  are  issues  and  discussion 
points  that  may  not  be  fully  resolved  at  the  beginning  of  the  research  effort  and  will  involve 
some  investigation  during  the  course  of  the  research  in  order  to  resolve  them.  Two  such  areas 
have  been  identified  during  the  development  of  the  COMPASS  paradigm.  The  first  is  the 
deviation  from  traditional  ethnographic  qualitative  techniques  that  have  been  taken  while  still 
operating  under  the  guise  of  using  ethnographic  methodologies.  The  second  is  the  potential  in 
this  multi-simulation  process  to  generate  a  phenomenon  that  is  described  as  “stacking  of  errors.” 
This  is  a  situation  where  the  inherent  error  induced  by  each  stage  of  the  process  from  data 
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collection  to  neural  network  simulation  to  discrete  event  simulation  is  carried  forward  to  the  next 
stage  to  the  extent  that  the  error  is  significant  in  the  final  result.  This  is  a  challenge  to  the 
validation  of  the  overall  process. 

5.2.1.  Challenges  to  Validation. 

While  the  COMPASS  framework  provides  for  controlling  the  error  generated  at  each 
stage  of  the  process,  there  is  some  concern  that  cumulative  error  across  the  entire  framework 
may  become  unmanageable.  Current  strategies  for  controlling  error  during  the  COMPASS 
process  begin  at  the  data  collection  stage.  There  is  a  strict  adherence  to  quantitative  recording  of 
observations  in  the  field.  For  those  events  that  are  binary  or  categorical  in  nature  it  is  a  simple 
matter  of  determining  if  the  event  is  active  or  which  category  of  a  set  of  options  is  appropriate  at 
the  time  of  observation.  For  those  events  that  are  recorded  as  continuous  variables  many  are 
completely  objective  in  nature,  however  some  are  subjective.  For  example,  ambient  temperature 
and  noise  levels  can  be  recorded  exactly  (at  least  to  the  accuracy  of  the  recording  instruments.) 
However,  for  those  events  that  require  a  subjective  assessment  of  the  observer  there  is  the 
increased  risk  of  error.  For  example,  if  the  observer  has  to  judge  the  level  of  usage  of  a 
communications  system  on  a  scale  of  0-100%  at  the  time  of  observation,  then  error  might  be 
induced  where  another  observer  might  make  a  different  determination.  This  type  of  error  is 
minimized  in  COMPASS  by  requiring  the  assessment  to  be  made  on  the  spot  at  the  time  of 
observation  when  all  the  sensory  input  conditions  of  the  environment  can  be  experienced  by  the 
observer.  This  is  as  opposed  to  making  the  judgment  at  a  later  time  from  the  analysis  of  a  video 
recording  of  the  events  when  only  the  video  image  is  available  to  the  observer  in  order  to  make 
the  judgment. 

During  the  neural  network  analysis  error  can  be  induced  from  the  fact  that  the  exact 
nature  of  the  architecture  of  the  network  is  up  to  the  subjective  choice  of  the  analyst.  In  this  case 
a  multi-layer  perceptron  approach  was  chosen  as  being  one  of  the  most  widely  accepted 
architectures  for  this  type  of  analysis.  This  mandated  the  use  of  one  input  and  one  output  layer 
with  the  number  of  nodes  in  each  exactly  matching  the  IV  and  DV  of  the  observed  data. 
However,  the  number  of  hidden  layers  and  number  of  nodes  within  the  hidden  layer  is  purely  up 
to  the  discretion  of  the  analyst.  The  desire  is  to  have  as  few  a  number  of  hidden  layers  with  each 
hidden  layer  containing  as  few  number  of  nodes  required  to  achieve  successful  training.  For  the 
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number  of  hidden  layers  the  logical  starting  point  for  the  COMPASS  analysis  was  one.  For  the 
number  of  nodes  within  it  the  number  of  90  was  chosen  as  being  between  25-50%  of  the  number 
of  input  nodes  which  agrees  with  opinion  stated  in  the  literature.  An  excessive  number  of  hidden 
layers  or  hidden  nodes  can  increase  the  computational  result  of  having  the  network  attempt  to 
model  noise  within  the  data.  Two  few  hidden  nodes  or  layers  can  prevent  the  network  from 
being  able  to  successfully  train  itself  because  of  insufficient  computational  capability  which 
resides  primarily  within  the  hidden  nodes.  In  many  cases  the  determination  of  numbers  of 
hidden  nodes  and  layers  can  only  be  determined  through  iterative  network  simulation  runs  where 
the  hidden  node  numbers  are  varied  between  runs.  For  the  analysis  presented  thus  far  in  this 
research  successful  training  was  achieved  with  very  low  epoch  and  pruning  replication  counts. 
The  results  supported  the  conclusion  that  the  network  was  operating  successfully  with  no 
overtraining  with  its  resultant  potential  for  modeling  error  through  network  noise. 

During  the  discrete  event  simulation  phase  error  can  be  induced  through  the  effects  of 
random  variability  from  the  random  number  generation  process.  These  effects  were  countered  in 
COMPASS  through  a  numerical  determination  of  the  number  of  replication  cycles  required  to 
run  the  simulation  in  order  to  achieve  a  confidence  interval  of  95%. 

While  attempts  to  control  individual  error  effects  have  been  carefully  managed  during 
this  process,  the  cumulative  effects  of  error  that  is  insignificant  at  each  stage  can  combine  with 
error  generated  during  subsequent  stages  to  produce  a  cumulative  error  effect,  also  described  as 
“stacking  of  errors,”  that  is  significant.  This  is  an  issue  for  future  investigation  and  analysis.  For 
the  current  effort  this  effect  has  been  minimized  through  as  close  as  possible  adherence  to 
procedures  at  each  computational  stage. 

5.2.2.  Ethnography  Challenges. 

Conventional  ethnography  theory,  as  fonnulated  by  Nardi  and  others  (Bonnan  et  ah, 

1986;  Lawlor  and  Mattingly,  2001;  LeCompte  and  Goetz,  1982;  Nardi,  1997b;  Petty,  1997; 
Segall,  1991;  Uzzell,  2000),  is  highly  qualitative  in  nature  where  the  observer  becomes  immersed 
in  the  culture  of  the  ethnic  group  being  observed  while  maintaining  as  objective  an  opinion  as 
possible  in  interpreting  the  meaning  and  nature  of  observations  made.  Those  espousing  this 
discipline  would  also  postulate  that  all  situations  are  different  and  should  be  considered 
differently  even  if  they  involve  individuals  perfonning  like  activities.  For  example,  if  baseball 
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teams  were  being  observed  to  try  to  ascertain  what  contributes  to  success  on  the  field,  they  might 
be  observed  playing  during  the  day  or  at  night,  and  in  different  stadiums  of  varying  quality  of  the 
facilities.  True  adherence  to  ethnographic  principles  would  consider  each  of  these  conditions  as 
unique  and  require  a  separate  analysis  for  each  case.  Other  researchers  might  take  a  different 
approach  with  the  view  that  baseball  is  baseball  and  that  it  can  be  observed  as  such  regardless  of 
the  circumstances. 

The  COMPASS  paradigm  takes  a  modified  approach  to  this  discipline  by  promulgating  a 
responsible  approach  that  adheres  to  ethnographic  constructs  as  closely  as  possible  while 
evaluating  a  domain  that  is  so  complex  as  to  preclude  the  ability  to  assess  the  meaning  of 
hundreds  of  qualitative  assessments  for  each  possible  condition  that  might  be  observed.  In  order 
to  reduce  potential  field  observational  error  as  previously  described,  a  strict  quantitative 
approach  to  observation  recording  is  utilized  that  requires  on  the  spot  determination  of  the  values 
of  not  only  digital  and  interval  variables,  but  also  continuous  measures  of  perfonnance.  The  key 
to  the  ethnographic  approach  in  COMPASS  is  not  the  way  data  is  recorded,  but  the  way  it  is 
gathered.  Achieving  the  confidence  and  respect  of  the  target  audience  is  one  important  aspect. 
This  is  achieved  by  having  an  observer  that  is  knowledgeable  in  the  activities  being  performed  to 
the  extent  that  they  can  speak  the  technical  dialect  of  the  group  that  will  almost  always  exist 
especially  in  military  circles.  They  must  be  able  to  demonstrate  that  the  group  does  not  have  to 
deal  with  them  at  a  level  of  explaining  the  basic  nature  of  the  environment,  but  can  communicate 
with  them  as  perceived  peers  to  reflect  why  and  how  they  do  the  things  they  do.  The  successful 
integration  of  ethnographic  considerations  into  a  field  observation  situation  that  is  only 
temporary  in  nature  with  a  high  degree  of  complexity  forces  more  stringent  data  recording 
techniques.  This  is  one  of  the  things  that  makes  the  COMPASS  framework  unique. 

5.3.  Topics  For  Future  Research. 

This  project  has  the  potential  to  assist  the  Army  in  many  different  ways  with  the  new 
COMPASS  paradigm  forming  the  basis  for  continued  investigative  work.  The  focus  chosen  for 
this  dissertation  is  the  first  logical  step  to  achieve  this  overall  objective  of  providing  a  foundation 
for  a  variety  of  future  scientific  research  efforts  not  only  for  the  Army,  but  also  for  investigating 
complex,  time  critical  C2Ss,  in  general. 
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As  the  work  started  by  this  dissertation  continues  in  the  future  it  is  anticipated  that  other 
research  areas  that  have  potential  contribution  to  the  overall  project  of  developing  ways  and 
means  to  improve  TOC  performance  will  be  identified  as  candidates  for  further  investigation. 
Some  of  these  anticipated  topics  relate  to  physiological  and  psychophysical  stressors  related  to 
trying  to  perfonn  HCI  tasks  while  bouncing  around  in  the  back  of  a  tracked  command  post 
vehicle  that  is  moving  over  open  terrain.  Other  research  could  focus  on  training  issues  related  to 
fewer  numbers  of  operators  and  decision  makers  attempting  to  control  greater  numbers  of 
resources  on  the  battlefield.  Another  area  that  has  been  previously  evaluated  is  the  effect  on 
human  operators  from  prolonged  exertion  in  confined  environments  such  as  the  inside  of  military 
tactical  vehicles  (Hicks,  1973;  Hicks,  1960a;  Hicks,  1960b;  Hicks,  1961a;  Hicks,  1961b;  Hicks, 
1961c;  Hicks,  1962;  Hicks,  1963;  Hicks,  1964a;  Hicks,  1964b).  This  work  could  be  expanded 
with  a  look  at  the  efficiency  of  HCI  performance  under  these  conditions  that  could  also  include 
sleep  deprivation  and  exposure  to  temperature  extremes.  Gender  issues  from  opposite-sex 
operators  working  in  close  quarters  for  extended  periods  of  time  could  also  be  the  basis  for  TOC 
performance  changes  and  ways  to  alleviate  these  issues  could  be  investigated. 

Other,  more  traditional  techniques,  that  could  be  used  to  refine  the  COMPASS 
framework  include  the  selection  of  designs  that  are  either  consist  of  pure  factorial  designs  or 
some  combination  of  factorial,  blocked,  or  central  composite  methodologies  (Clark  and  Williges, 
1972a;  Clark  and  Williges,  1972b;  Simon,  1970;  Williges,  1976;  Williges  and  Baron,  1972; 
Williges  and  Mills,  1972).  The  linear  and  higher  order  components  of  the  IV  responses  could 
then  be  considered  according  to  their  response  surfaces  for  a  more  complete  look  at  overall 
system  response  (Simon,  1968;  Simon,  1976;  Williges  and  Simon,  1970;  Williges  and  Simon, 
1971). 

5.3.1.  Future  Enhancements  to  COMPASS. 

This  dissertation  is  considered  just  the  first  phase  and  what  is  to  be  a  long  term  research 
effort  investigating  the  perfonnance  of  C2Ss.  While  this  effort  has  used  tools  such  as  JNNS  for 
neural  network  analyses  and  CoHOST  for  TOC  evaluation,  there  are  more  sophisticated  and 
newer  technology  approaches  that  could  not  be  explored  here  due  to  time  and  budget  constraints. 
Future  work  will  hopefully  be  able  to  capitalize  on  more  sophisticated  neural  network 
simulations  and  next  generation  discrete  event  approaches  to  TOC  simulation.  Considerations  in 
these  two  topical  areas  are: 
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5 .3 . 1 . 1 .  Improving  the  Neural  Network  Interface. 

While  perfonning  the  neural  network  phase  there  were  two  neural  network  simulation 
packages  that  were  evaluated.  The  freeware  JNNS  simulation  used  here  has  been  discussed 
previously.  While  it  performs  well  during  the  network  training  and  pruning  stages,  it  has  no 
ability  to  directly  translate  its  internal  programming  to  a  form  that  can  be  used  as  an  interface  to 
other  simulations.  In  addition,  it  is  very  manually  intensive  to  use  JNNS  as  all  of  the  data 
translation,  separation  of  data  into  training  and  validation  data  sets,  and  layout  of  the  network 
must  be  performed  separately  and  put  into  the  required  format  necessary  to  interface  with  JNNS. 
Other  packages  allow  for  the  generation  of  C++  language  code  that  describe  the  relationships 
between  the  input  and  output  nodes  and  can  be  used  as  models  of  that  relationship. 

One  software  product  that  provides  this  capability  is  the  Enterprise  Miner  (EM)  program 
from  SAS  Institute.  EM  is  a  suite  of  datamining  modules  that  includes  regression  analysis, 
cluster  analysis,  decision  tree  analysis,  neural  network  analysis,  and  principle  components 
analysis.  There  are  also  automated  tools  that  allow  for  partitioning,  attribute  specification  and 
validation  of  data.  Each  module  is  invoked  and  linked  to  the  other  modules  through  the  use  of  a 
GUI  based  tools  diagram.  Figure  40  shows  an  example  of  an  EM  tool  control  diagram  that  was 
used  in  its  evaluation  here. 
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Figure  40  -  Enterprise  Miner  Tools  Diagram 

To  use  this  simulation,  parameters  are  entered  for  each  module  and  the  simulation 
performs  a  full  preparation  and  analysis  of  the  data  during  each  run.  Thus,  this  type  of  system 
could  have  great  potential  for  applications,  like  the  COMPASS  paradigm,  where  the  database  is 
constantly  growing  with  each  exercise  observation.  The  neural  network  is  represented  in  a 
diagram  very  similar  to  INNS  as  shown  in  Figure  41,  however,  EM  creates  this  diagram 
automatically  from  the  dataset  while  JNNS  requires  that  the  diagram  be  manually  entered  and 
configured  to  establish  the  network. 
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Figure  41  -  EM  Neural  Network  Diagram 

EM  also  determines  the  point  of  full  training  automatically  and  stops  the  simulation  at 
that  point  thus  preventing  any  possibility  of  overtraining.  An  error  curve  of  the  training  process 
is  provided  as  shown  in  Figure  42,  however,  it  is  for  informational  purposes  only  as  no 
interpretation  is  required  of  the  analyst  to  determine  the  number  of  epochs  required  to  achieve 
full  training. 
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Figure  42  -  EM  Error  Graph 

Once  the  full  datamining  procedure  is  coded  into  EM,  then  an  analysis  can  be  performed 
after  each  exercise  observation  with  minimal  manual  preparation  required.  However,  the  power 
of  this  simulation  also  includes  the  ability  to  translate  the  logic  from  the  neural  network,  or  any 
other  simulation  module,  into  SAS  language  code  or  C++  code  using  the  Score  and  C*Score 
nodes. 

The  SAS  EM  package  was  not  selected  at  this  time  because  of  technical  difficulties  with 
its  use  that  could  not  be  resolved  in  time  to  support  this  analysis  and  because  of  the  cost  of  the 
program  as  limited  funding  was  available.  However,  its  potential  for  future  analyses  is 
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significant  and  could  become  the  neural  network  of  choice  to  support  ongoing  studies  involving 
a  constantly  growing  database. 

5.3. 1 .2.  Improving  the  CoHOST  Operator  Interface-C3  TRACE. 

The  CoHOST  discrete  event  simulation  was  the  product  of  a  multi-year  effort  involving 
considerable  expense  and  manpower  as  previously  described.  However,  it  is  a  very  structured 
simulation  model  not  well  suited  for  evaluation  of  system  changes.  In  2001,  recognizing  the 
need  for  a  rapid  turnaround  capability  to  examine  TOC  operations  and  structures,  the  U.S.  Anny 
developed  a  GUI  shell  to  reside  on  top  of  the  Micro  Saint™  discrete  event  programming 
language  that  provides  an  ability  to  quickly  configure  and  test  TOC  structures  using  a  building 
block  approach  for  the  identification  of  operators,  operator  performance  characteristics,  and  TOC 
system  elements.  This  GUI  interface  is  called  Command,  Control  and  Communication- 
Techniques  for  Reliable  Assessment  of  Concept  Execution  (C3-TRACE)  (Plott,  2002).  The 
CoHOST  simulation  has  already  been  rehosted  into  C3-TRACE  and,  once  testing  and  validation 
is  complete,  is  expected  to  support  future  COMPASS  efforts. 

Operator  workload  is  determined  by  a  visual,  auditory,  cognitive,  and  psychomotor 
(VACP)  (McCracken  and  Aldrich,  1984)  assessment  of  operator  activity  multiplied  by  the 
amount  of  time  that  the  tasks  are  being  performed.  The  C3  TRACE  graphical  user  interface 
(GUI)  to  CoHOST-R  allows  for  the  definition  of  PSF  factors  such  as  aptitude,  age,  and  level  of 
training  that  the  simulation  then  uses  to  modify  actual  task  perfonnance  times.  These  embedded 
factors  can  be  used  in  the  PSF  definition  or  user  defined  factors  may  be  established  and  used.  By 
taking  the  results  of  the  regression  analysis  of  the  INNS  products  and  mapping  them  to  task  time 
and  follow-on  workload  on  the  operator  a  set  of  user  defined  PSF  factors  can  be  defined  that  will 
cause  the  amount  of  time  required  to  perform  tasks  across  the  TOC  model.  The  simulation’s 
work  and  task  load  performance  then  becomes  responsive  to  C2  system  mission  task  demands  as 
they  result  in  subsequent  operator  task  time  performance. 

Figure  43  shows  an  example  of  the  GUI  window  used  to  define  the  operators  and  sections 
of  a  TOC  using  C3  TRACE.  Parameters  for  the  operators  and  other  structures  of  the  TOC  and 
configured  with  similar  graphical  windows.  The  scenario  is  provided  through  a  series  of 
communications  events  that  are  loaded  into  a  data  table. 
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Figure  43  -  C3  TRACE  Organization  Tree  Window 


After  all  the  data  structures  have  been  identified  the  performance  logic  is  entered  as  a 
Micro  Saint™  like  logic  flow  diagram  that  identifies  performance  activities  for  the  TOC  as 
shown  in  Figure  44. 
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Figure  44  -  C3  TRACE  Task  Definitions 

Upon  execution  C3  TRACE  automatically  generates  a  Micro  Saint™  simulation  model 
of  the  TOC  based  on  the  C3  TRACE  definitions  and  executes  it  with  an  embedded  Micro  Saint™ 
simulation  engine.  Thus,  conceptual  changes  can  be  made  in  the  C3  TRACE  definitions  and  a 
new  model  can  be  generated  and  executed  that  provides  data  for  immediate  analysis. 

C3  TRACE  was  evaluated  for  use  in  this  dissertation  effort,  however,  being  a  new 
product  still  in  the  final  stages  of  development  it  could  not  provide  reliable  models  in  time  for 
this  study.  However,  its  potential  for  support  of  evolving  COMPASS  studies  where  the  discrete 
event  simulations  need  to  have  the  ability  for  rapid  modification  to  support  TOC  analyses  is 
significant.  Future  work  in  TOC  simulations  will  be  able  to  greatly  benefit  from  rapid  generation 
development  packages  such  as  C3  TRACE. 

5.3.2.  Future  Evolution  of  the  TOC  Knowledge  Base. 

This  research  has  developed  the  COMPASS  paradigm  with  data  gathered  from  the 
observation  of  three  TOC  deployments.  These  three  deployments  have  provided  an  opportunity 
to  observe  each  of  the  three  major  types  of  brigade  training  exercises.  These  three  types  have 
been  previously  described  as  a  test  exercise,  an  evaluated  exercise,  and  a  pure  training  exercise. 
One  of  these  was  a  field  training  exercise  while  the  other  two  were  conducted  using  computer 
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simulation  to  replicate  the  movement  of  forces  on  the  battlefield.  While  all  three  of  these 
exercises  were  conducted  at  Fort  Hood,  Texas,  it  is  envisioned  that  future  TOC  observations  will 
continue  to  expand  the  data  on  each  of  these  exercise  types  with  TOC  deployments  across  the 
continental  United  States.  Brigades  are  in  constant  rotation  through  the  National  Training  Center 
(NTC)  at  Fort  Irwin,  California  conducting  full  scale  maneuver  exercises  that  are  evaluated. 
National  Guard  brigades  across  the  nation  are  continually  conducting  evaluated  exercises  using 
computer  simulation  where  evaluator  teams  from  the  Battle  Command  Training  Program’s 
(BCTP)  Operations  Group  C  deploys  from  Fort  Leavenworth,  Kansas,  along  with  the  computer 
simulation  equipment,  to  conduct  evaluated  exercises  at  the  National  Guard  brigade’s  home 
location.  In  addition  to  Fort  Hood,  there  are  major  combat  forces  stationed  at  posts  around  the 
country.  BCTP  exercises  are  in  continued  execution  with  Operations  Groups  A  and  B  for  them 
as  well  as  exercises  that  they  conduct  in  preparation  for  NTC  rotations. 

All  of  these  exercise  events  provide  a  rich  environment  for  observation  of  future  TOC 
deployments.  It  is  the  goal  of  this  research  project  to  observe  30  to  40  or  more  TOCs  over  the 
next  few  years  and  grow  the  TOC  observation  knowledge  base  to  be  truly  capable  of  supporting 
studies  on  what  is  and  is  not  important  to  TOC  operation  and  design. 

5.4.  Conclusions. 

This  dissertation  has  developed  a  procedural  and  analytical  framework  that  allows  for  the 
quantitative  description  of  the  complete  C2S.  While  individual  and  team  performance  is  a  major 
component  of  the  overall  system,  it  is  by  no  means  the  only  component.  Although  the  literature 
acknowledges  the  existence  of  a  command  and  control  “system,”  the  research  predominately 
focuses  on  the  human  computer  interface  (HCI)  and  team  performance.  The  military  decision 
making  process  is  a  structured  approach  to  problem  solving,  however,  there  are  situations  where 
it  does  not  work  very  well  and  research  that  just  focuses  on  this  process  does  not  account  for 
other  variables  that  can  induce  stress  and  affect  the  situation  awareness  of  the  decision  makers. 
The  COMPASS  approach  to  analysis  of  the  TOC  and  the  way  it  operates,  encompasses  all  these 
internal  variabilities  as  it  considers  this  system  from  a  top  level  viewpoint  while  still  maintaining 
significant  levels  of  detail  concerning  the  processes  that  are  going  on  between  humans  and  other 
humans  and  between  humans  and  the  system  as  represented  by  the  HCI  interface. 
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The  overall  goal  of  this  study  has  been  to  develop  an  observation  and  analysis  technique 
that  will  enable  a  better  understanding  of  the  systemic  performance  of  U.S.  Army  TOCs.  This 
technique  is  validated  using  exploratory  observations  of  selected  TOCs  in  operation  that 
generates  data  that  is  be  evaluated  in  an  existing  computer  simulation  chosen  for  its  ability  to 
address  the  different  activities  present  in  a  modern  day  TOC.  Each  future  round  of  observation, 
analysis,  and  simulation  will  serve  to  refine  and  update  the  computer  simulation  and  provide  an 
evolving  evaluative  tool  that  will  be  utilized  to  investigate  future  TOC  systems  while  they  are 
still  in  the  design  stage. 
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Appendix  A  -  TOC  Data  Collection  Spreadsheet. 


Critical  Battlefield  Tasks 

Data  Collection  Spreadsheet 

Command  and  Control  System  TOC  Operations 

Central  Standard  Time  (CST)  - 

4/23/2002  7:00 

4/23/2002  8:00 

4/23/2002  9:00 

Hour  Of  The  Day: 

7 

8 

9 

Day  Of  The  Exercise: 

1 

1 

1 

Greenwich  Mean  Time  (GMT)  -  ZULU 

GMT  =  CST  +  5 

4/23/2002  12:00 

4/23/2002  13:00 

4/23/2002  14:00 

Type  of  TOC  -  Analog  (0),  Digital  (1) 

1 

1 

1 

Size  of  TOC:  (0)-  Battalion;  (1)-  Brigade 

1 

1 

1 

Type  of  Exercise: 

1- OTC  Test 

2-  BCTP  Evaluation 

3-  NTC  Rotation 

4-  Standalone  Training 

1 

1 

1 

Type  of  Simulation  Support: 

0-  None 

1-  JANUS 

2-  JANUS/STORM 

3-  CBS 

4-  CCTT 

5-  STORM 

1 

1 

1 

Primary  Task 

Select  Task  Number  From  List: 

1  Departure  from  the  assembly  area. 

2  Passage  of  lines. 

3  Movement  to  the  line  of  departure. 

4  Breach  of  main  obstacle  belt. 

5  Penetration  of  defensive  positions. 

6  Reaction  to  counterattack  forces. 

7  River  crossing. 

8  Seizure  of  key  terrain. 

9  Seizure  of  objective. 

10  Destruction,  capture,  or  bypass  of  enemy  force. 

11  Fixing  enemy  in  position. 

12  Synchronization  with  supporting  forces. 

13  Use  of  reserves. 

14  Deep  operations. 

15  Destruction  of  first  echelon  forces. 

16  Destruction  of  follow-on  forces. 

17  Commitment  of  counterattack  forces. 

18  Deception  activities. 

19  Rear  operations. 

20  Entry  into  area  of  operations. 

21  Peacekeeping  operations. 

22  Transfer  of  mission. 

20 

20 

20 

Secondary  Task 

Select  Task  Number  From  List:  1  Departure  from  the  assembly  area.  2  Passage  of 

Ancillary  Task 

Select  Task  Number  From  List:  1  Departure  from  the  assembly  area.  2  Passage  of 
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Battlefield  Operating 
Systems  (BOS) 

Intelligence 

Battlefield  Function  (BF)  - 

Indicate  Which  BF  Is  Being  Performed 

1  Conduct  Intelligence  Planning 

2  Collect  Information 

3  Process  Information 

4  Disseminate  Intelligence 

Maneuver 

Battlefield  Function  (BF)  - 

Indicate  Which  BF  Is  Being  Performed 

1  Conduct  Tactical  Movement 

2  Engage  Enemy  with  Direct  Fire  and  Maneuver 

Fire  Support 

Battlefield  Function  (BF)  - 

Indicate  Which  BF  Is  Being  Performed 

1  Employ  Mortars 

2  Employ  Field  Artillery 

3  Employ  Close  Air  Support 

4  Conduct  Electronic  Collection  and  Attack 

5  Conduct  PSYOP 

6  Employ  Chemical  Weapons 

7  Conduct  Counter  Target  Acquisition  Operations 

8  Employ  Naval  Surface  Fires 

9  Coordinate,  Synchronize,  and  Integrate  Fire  Support 

> 

Mobility  and  Survivability 

Battlefield  Function  (BF)  - 

Air  Defense 

Battlefield  Function  (BF)  - 

Indicate  Which  BF  Is  Being  Performed 

1  Take  Active  Air  Defense  Measures 

2  Take  Passive  Air  Defense  Measures 

Command  and  Control 

Battlefield  Function  (BF)  - 

Indicate  Which  BF  Is  Being  Performed 

1  Plan  for  Combat  Operations 

2  Direct  and  Lead  Unit  During  the  Preparation  Phase  of  the  Battle 

3  Direct  and  Lead  Unit  in  Execution  of  Battle 

>  2  2  1 

Communications 

Rate  usage  of  Each  System:  1-10,  10  Highest 

1 

AFATDS  -  Advanced  Field  Artillery  Tactical  Data  System 

0 

0 

1 

Rate  each  system 

2 

AMPS  -  Aviation  Mission  Planning  System 

on  a  scale  of  1  (low) 

3 

Applique 

to  10  (high)  as  to  its 

4 

ASAS  -  All  Source  Analysis  System 

1 

1 

1 

level  of  observed  usage 

5 

CTIS  -  The  Combat  Terrain  Information  System 

6 

CSSCS  -  The  Combat  Service  Support  and  Control  System 

7 

FAADC2  -  Forward  Area  Air  Defense  Command  and  Control 

8 

FBCB2  -  Force  XXI  Battle  Command  Brigade  and  Below 

1 

1 

9 

IMETS  -  Integrated  Meteorological  System 

10 

MCS/P  -  Maneuver  Control  Systems/Phoenix 

1 

1 

11 

AMDWS,  ADA 

12 

CGS.  JSTARS 
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Intelligence  Collection 

Rate  usage  of  Each  System:  1-10,  10  Highest 

1 

TUAV  -  Tactical  Unmanned  Aerial  Vehicle 

2 

JSTARS  -  Joint  &  Strategic  Tactical  Airborne  Radar  System 

3 

GSR  -  Ground  Surveillance  Radar 

Other  Factors 

Rate  usage  of  Each  System:  1-10,  10  Highest 

1 

Battle  Timing 

2 

Battle  Tempo 

3 

Reconnaissance  Operations 

4 

Information  Operations 

5 

Tactics 

6 

Observed  Activity  /  Stress  Level  in  TOC 

Personnel 

111 

Indicate  Present  (1)  or  Absent  (0)  in  TOC  - 

1 

Commanding  Officer  -  Col.  Campbell 

0 

0 

Executive  Officer 

0 

0 

Battle  Captain 

1 

1 

S3  -  Major  Winkle 

0 

0 

S3  RTO 

S2  -  Cpt.  Boone  /  Cpt.  Briggman 

1 

0 

FSO  -  Cpt  Ikena 

1 

1 

Engineer 

1 

1 

Environmental  Conditions 

Record  Actual  Readings 

Sky  Condition:  Clear-1;  Overcast-2;  Rain-3 

Wind  Condition:  0-  none;  1-  low;  2-  moderate;  3-  high 

o 

0 

0 

Light  Level,  Foot  Lamberts  -  Background  at  20  ft. 

o 

z 

2 

2 

Light  Level,  Foot  Lamberts  -  Map  Display  (Rear  Projection)  at  20  ft. 

20 

20 

10 

Noise,  dbA 

66 

66 

66 

Dry  Bulb  Temperature,  degrees  F 

74 

74 

77 

Wet  Bulb  Temperature,  degrees  F 

70 

70 

71 

Relative  Humidity,  % 

82 

82 

75 
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Appendix  B  -  Glossary  Of  U.S.  Army 
Acronyms. 


(D) 

Draft 

(M) 

(Mechanized) 

(T) 

Towed 

“Heart  Beat” 

Futuristic  Concept  Of 

Monitoring  The  Body  Vitals  Of 
A  Soldier. 

1BCT 

1st  Brigade  Combat  Team 

1BCT/4ID 

1st  Brigade  Combat  Team  (4th 
infantry  Division,  Ft.  Hood,  TX) 

1LT 

1st  Lieutenant 

1SG 

First  Sergeant 

2S1 

Russian  Built  Self  -Propelled 
Light  Artillery 

2S6 

Russian  Built  Self-Propelled 
Medium  Artillery 

96  hours 

After  96  Hours,  The  FCS  Must 
Be  Fully  Deployed. 

A/DACG 

Arrival/Departure  Airtleld 
Control  Group 

AIL 

Administrative  And  Logistics 

A/SPOD 

Aerial/Sea  Ports  Of  Debarkation 

A2 

Antiarmor 

A2C2 

Army  Airspace  Command  And 
Control 

A2C2S 

Army  Aviation  Command  And 
Control  System 

AA 

Assembly  Areas 

AA 

Active  Army 

AAC 

Anti-Aircraft 

AAC 

Army  Acquisition  Corps 

AADC 

Area  Air  Defense  Commander 

AAE 

Army  Acquisition  Executive 

AAMMP 

Active  Army  Military  Manpower 
Program 

AAO 

Army  Acquisition  Objective 

AAR 

After-Action  Review 

AASA 

Administrative  Assistant  To  The 
Secretary  Of  The  Army 

AASLT 

Air  Assault 

AAV 

Army  Aviation 

AAVs 

Autonomous  Air  Vehicle  (See 
Also  FAAV) 

AAW 

(1)  Antiair  Warfare;  (2)  Army 
Acquisition  Workforce 

AAW 

Analysis  And  Wargaming 

AB2 

Army  Brigade  And  Below 

Abb 

Abbreviated,  In  This  Use,  A 
Report  With  Only  Minimal 
Information 

ABCS 

Army  Battle  Command  System 

ABE 

Assistant  Brigade  Engineer 

ABF 

Attack  By  Fire 

ABGD 

Air  Base  Ground  Defense 

ABMS 

Assault  Breach  Marking  System 

ABN 

Airborne 

AC 

Active  Component 

AC 

Aircraft 

ACA 

Airspace  Control  Authority 

ACA 

Air  Coordination  Area 

ACAT 

Acquisition  Category 

ACC 

Air  Component  Commander 

ACC 

Army  Commanders’  Conference 

ACC 

Army  Component  Commander 

ACE 

Analysis  Control  Element; 

ACE 

Advanced  Collaborative 
Environment 

ACMR 

Air  Control  Measure  Request 

ACO 

Airspace  Control  Order 

ACR 

Armored  Cavalry  Regiment 

ACR 

Armored  Cavalry  Regiment 

ACSIM 

Assistant  Chief  Of  Staff  For 
Installation  Management 

ACT 

Analysis  Control  Team 

ACT  II 

Advanced  Concepts  & 
Technology  Program  II 

ACTD 

Advanced  Concept  Technology 
Demonstration 

ACTS 

Army  Criteria  Tracking  System 

ACTY 

Activity 

ACU 

Area  Common  User 

ACUS 

Army  Common  User  System 

AD 

Air  Defense 

AD 

Air  Defense  Or  Armored 
Division  (Depending  On  Usage) 

ADA 

Air  Defense  Artillery 

AD  ADO 

Assistant  Division  Air  Defense 
Officer 

ADAM 

Air  Defense  Air  Management 

ADAM/RAAM 

Aerial  Denial  Artillery 
Munition/Remote  Antiarmor 
Minefield 

ADC 

Area  Damage  Control 

ADCOORD 

Air  Defense  Coordinator 

ADCSOPS 

Assistant  DCSOPS 

ADDS 

Automatic  Data  Distribution 
System 

ADE 

Air  Defense  Element 

ADE 

Assistant  Division  Engineer 

ADHPM 

Artillery-Delivered,  High- 
Precision  Munitions 

ADJ 

Adjustment 

ADM 

Analytical  Decision-Making 
Model 

ADM 

Acquisition  Decision 
Memorandum 

ADMIN 

Administration 

ADMS 

Area  Denial  Munition  System 
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ADOCS 

Automated  Deep  Operations 

AMEDDC&S 

Army  Medical  Department 

Coordination  System 

Center  And  School 

ADT 

Active  Duty  For  Training 

AMIM 

Army  Modernization  Information 

ADWCS 

Air  Defense  Weapons  Control 

Memorandum 

Status 

AMO 

Aviation  Medical  Officer 

AE 

Aerial  Exploitation 

AMOPES 

Army  Mobilization  And 

AF 

US  Air  Force 

Operations  Planning  And 

AFAS 

Advanced  Field  Artillery  System 

Execution  System 

AFATDS 

Advanced  Field  Artillery 

AMOPES 

Army  Mob.  And  Ops  Planning 

Tactical  Data  System 

And  Execution  System 

AFC 

Army  Fiscally  Constrained 

AMOPES 

Army  Mobilization  And 

AFCF 

Army  Fiscally  Constrained  Force 

Operations  Planning  And 

AFCS 

Automatic  Fire  Control  System 

Execution  System 

AFH 

Army  Family  Housing 

AMP 

Army  Modernization  Plan;  Army 

AFHO 

Family  Housing,  Army 

Mobilization  Plan 

(Operations) 

AMPMOD 

Army  Materiel  Plan  Modified 

AFPDA 

Army  Force  Planning  Data  And 

AMRD 

Army  Modernization  Reference 

Assumptions 

Data 

AFSE 

Automated  Fire  Support  Element 

AMSAA 

Army  Materiel  Systems  Analysis 

AFV 

Airborne  Fighting  Vehicle 

Agency 

AG 

Adjutant  General 

AMSCO 

Army  Management  Structure 

AGCCS 

Army  Global  Command  And 

Code 

Control  System 

AMT 

Army  Modernization  Training 

AGL 

Above  Ground  Level 

ANAL 

Analysis 

AGMB 

Advance  Guard  Main  Body 

ANGLICO 

Air  And  Naval  Gunfire  Liaison 

AGR 

Active  Guard  Reserve 

Company 

AGS 

Armored  Gun  System 

AO 

Area  Of  Operations 

AH 

Attack  Helicopter 

Ao 

Operational  Availability 

AH-64 

Apache  Attack  Helicopter  (US) 

AOC 

Air  Operations  Center 

AI 

Air  Interdiction 

AOLCM 

Army  Organizational  Life  Cycle 

AI 

Additional  Issue 

Model 

AID 

Agency  For  International 

AOP 

Army  Order  Of  Precedence 

Development 

AOR 

Area  Of  Responsibility 

AIN 

Army  Interoperability  Network 

AOS 

Accoustic  Overwatching  Sensor 

AIS 

Automated  Information  System 

APADS 

Advanced  Precision  Airborne 

AIT 

Automatic  Identification 

Delivery  System 

Technology 

APB 

Acquisition  Program  Baseline 

AIT 

Advanced  Individual  Training 

APC 

Armored  Personnel  Carriers 

ALACV 

Advanced  Light-Armament  For 

APERS 

Antipersonnel 

Combat  Vehicles 

APG 

Army  Program  Guidance 

ALBE 

Airland  Battlefield  Environment 

APGM 

Army  Program  Guidance 

ALBM 

Airland  Battle  Management 

Memorandum 

ALCC 

Airlift  Control  Center 

APICM 

Armor-Piercing  Improved 

ALO 

Air  Liaison  Officer 

Conventional  Munition 

ALO 

Authorized  Level  Of 

APKWS 

2.75  Rocket 

Organization 

APOBS 

Antipersonnel  Obstacle 

ALOC 

Air  Line  Of  Communications 

Breaching  System 

ALOC 

Administrative  &  Logisitics 

APPG 

Army  Preliminary  Programming 

Operations  Center 

Guidance 

ALRPG 

Army  Long  Range  Planning 

APPIS 

Army  POM  Preparation 

Guidance 

Instructions  Supplement 

AMC 

U.S.  Army  Materiel  Command 

Apr 

April 

AMC 

United  States  Army  Materiel 

APR 

Army  Procurement  Requirement 

Command 

APS 

Active  Protection  System; 

AMEDD 

Army  Medical  Department 

Advanced  Photographic  System 
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APS 

Army  Planning  System;  Army 

ASA(FM&C) 

ASA  (Financial  Management  & 

Prepositioned  Stocks 

Comptroller) 

APSES 

Army  Prepositioned  Stocks 

ASA(IL&E) 

ASA  (Installations,  Logistics, 

Equipment  Sets 

And  Environment) 

APSOP 

Army  Prepositioned  Stocks 

ASA(M&RA) 

ASA  (Manpower  And  Reserve 

Operational  Project 

Affairs) 

APSS 

Army  Prepositioned  Stocks 

ASA(RDA) 

ASA  (Research,  Development, 

Sustainment 

And  Acquisition) 

APU 

Auxiliary  Power  Unit 

ASAP 

As  Soon  As  Possible 

AQF 

Advanced  Quick  Fix 

ASARC 

Assistant  Sectary  Of  The  Army 

AR 

Army  Regulation 

Review  Committee 

ARB 

Army  Resources  Board 

ASARC 

Army  Systems  Acquisition 

ARBSG 

Army  Resources  Board  Support 

Review  Council 

Group 

ASAS 

All-Source  Analysis  System 

ARCOM 

Army  Command 

ASB 

Army  Science  Board 

ARCOM 

United  States  Army  Reserve 

ASCC 

Army  Service  Component 

Command 

Command 

ARDEC 

Armament  Research, 

ASCC 

Army  Service  Component 

Development,  &  Engineering 

Commander 

Center 

ASD 

Assistant  Secretary  Of  Defense 

ARES 

Advanced  Robotic  Engagement 

ASD(C3I) 

ASD  (Command,  Control, 

System 

Communications,  And 

ARFOR 

Army  Force 

Intelligence) 

ARFPC 

Army  Reserve  Forces  Policy 

ASD(FMP) 

ASD  For  Force  Management 

Committee 

Policy 

ARI 

Army  Research  Institute 

ASD(HA) 

ASD  For  Health  Affairs 

ARL 

(1)  Army  Research  Laboratory; 

ASD(RA) 

ASD  (Reserve  Affairs) 

(2)  Airborne  Reconnaisance  Low 

ASD(S&R) 

ASD  For  Strategy  And 

ARLO 

Air  Reconnaissance  Liaison 

Requirements 

Officer 

ASI 

Additional  Skill  Identifier 

ARM 

Antiradiation  Missile 

ASIOE 

Associated  Support  Items  Of 

ARM 

Anti-Radiation  Missile 

Equipment 

ARMT 

Armament 

ASIOEP 

Associated  Support  Items  Of 

ARNG 

Army  National  Guard 

Equipment  And  Personnel 

ARNG-TSP 

Army  National  Guard-Troop 

ASIP 

Army  Stationing  And  Installation 

Structure  Program 

Plan 

ARNGUS 

Army  National  Guard  Of  The 

ASL 

Authorized  Stockage  List 

United  States 

ASLT 

Assault 

ARO 

Army  Research  Office 

ASM 

Armored  Systems  Modernization 

AROC 

Army  Requirements  Oversight 

ASOC 

Air  Support  Operations  Center 

Council 

ASPS 

All-Source  Production  System 

ARPA 

Advanced  Research  Projects 

ASST 

Advanced  Sensor/Submunition 

Agency 

Technology 

ARPERCEN 

Army  Reserve  Personnel  Center 

AST 

ATEC  System  Team 

ARPRINT 

Army  Program  For  Individual 

ASTAG 

Army  Science  And  Techonology 

Training 

Advisory  Group 

ARSEC 

Army  Secretariat 

A-STAMIDS 

Airborne  Standoff  Mine- 

ARSTAF 

Army  Staff 

Detection  System 

ARSTRUC 

Army  Structure  Message 

ASTMIS 

Army  Science  And  Techonology 

ART 

Army  Reserve  Technician 

Management  Information  System 

ARTEP 

Army  Training  And  Evaluation 

ASTMP 

Army  Science  And  Technology 

Program 

Master  Plan 

ARTY 

Artillery 

AT 

Antitank 

AS 

Acquisition  Strategy 

AT 

Annual  Training 

ASA 

Assistant  Secretary  Of  The  Army 

AT-5 

Russian-Built  Anti-Tank  Guided 

ASA(CW) 

ASA  (Civil  Works) 

Missile 
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ATACMS 

Army  Tactical  Missile  Systems 

AWE 

Advanced  Warfighting 

ATACMS 

Army  Tactical  Missile  System 

Experiment 

ATAS 

Air-To-Air  Stinger 

AWIS 

Army  Wwmccs  Information 

ATC 

Army  Training  Center 

System 

ATC 

Ammunition  Team  Chief 

AWP 

Annual  Work  Plan 

ATC 

Army  Training  Center 

AWR 

Army  War  Reserve 

ATCAS 

Advanced  Towed  Cannon 

AWRSA 

Army  War  Reserve  Stocks  For 

System 

Allies 

ATCCS 

Army  Tactical  Command  And 

AWRSI 

Army  War  Reserve  Secondary 

Control  System 

Items 

ATD 

Advanced  Technology 

AWWG 

Advanced  Warfighting  Working 

Demonstration 

Group 

ATD/C 

Aided  Target 

B2C2 

Battalion  And  Below  Command 

Detection/Classification 

And  Control 

ATEC 

United  States  Army  Test  And 

BAAV 

Brigade/Battalion  Autonomous 

Evaluation  Command 

Air  Vehicle 

ATEC 

Army  Test  And  Evaluation 

Backward  Capability  How  Will  This  System 

Command 

(New  Technology)  Be  Used  By 

ATOM 

Anti-Tank  Guided  Missile 

Legacy  Units 

ATK 

Attack 

BAE  155 

Name  Of  The  South  African 

ATK 

Attack  Position  Or  Attack 

Artillery  Piece  FCS  Is  Reviewing 

ATLS 

Advanced  Trauma  Life  Support 

ForNLOS. 

ATMCT 

Air  Terminal  Movement  Control 

BAS 

Battlespace  Awareness  Module 

Team 

BAT 

Brilliant  Anti-Tank 

ATMDE 

Army  Theater  Missile  Defense 

BC 

Bradley  Commander 

Element 

BC 

Battle  Captain  (Officer  In  Charge 

ATO 

Air  Tasking  Order 

Of  A  Command  Post) 

ATP 

Allied  Tactical  Publication; 

BC 

Battle  Command 

Ammunition  Transfer  Point 

BCC 

Battlefield  Circulation  Control 

ATRRS 

Army  Training  Requirements 

BCD 

Battle  Coordination  Detachment 

And  Resources  System 

(Formerly  Called  BCE,  See 

ATSA 

ATEC  Threat  Support  Activity 

Above) 

ATTN 

Attention 

BCE 

Battlefield  Coordination  Element 

AUEL 

Automated  Unit  Equipment  List 

BCIS 

Battlefield  Combat  Identification 

AUG 

August 

System 

AUGTDA 

Augmentation  TDA 

BCIS 

Battlefield  Combat  Identification 

AURS 

Automated  Unit  Reference  Sheet 

System 

AUTH 

Authorization 

BCOTM 

Battle  Command  On  The  Move 

AUTMV 

Automotive 

BCT 

Brigade  Combat  Team 

AUTO 

Automation 

BCTP 

Battle  Command  Training 

AUTS 

Automatic  Update  Transaction 

Program 

System 

BCV 

Bradley  Command  Vehicle 

AV 

Avenger 

BCV 

Battle  Command  Vehicle 

AV 

Air  Vehicle 

BD 

Battlefield  Distribution 

AVCSA 

Assistant  Vice  Chief  Of  Staff, 

BDA 

Battle  Damage  Assessment  / 

United  States  Army 

Bomb  Damage  Assessment 

AVIM 

Aviation  Intermediate 

BDCST 

Broadcast 

Maintenance 

BDE 

Brigade 

AVLB 

Armored  Vehicle  Launched 

BDO 

Battle  Dress  Overgarment 

Bridge 

BDP 

Battlefield  Development  Plan 

AVN 

Aviation 

BDU 

Battle  Dress  Uniform 

AVO 

Air  Vehicle  Operator 

BE 

Budget  Estimate 

AVUM 

Aviation  Unit  Maintenance 

BES 

Budget  Estimate  Supplement 

AWACS 

Airborne  Warning  And  Control 

BES 

Budget  Estimates  Submission 

System 

BEWSS 

Battlefield  Environment  Weapon 

AWCF 

Army  Working  Capital  Fund 

System  Simulation 
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BFA 

Battlefield  Functional  Area 

BY 

BFIST 

Bradley  Fire  Support  Team 

C2 

BFIST-V 

Bradley  Fire  Support  Team 

Vehicle 

C21 

BFM 

Battle-Scale  Forecast  Model 

C2V 

BFV 

Bradley  Fighting  Vehicle 

C2W 

BHL 

Battle  Handover  Line 

C3 

BIDS 

Biological  Integrated  Detection 

System 

C31 

BII 

Basic  Issue  Items 

BIM 

Battlefield  Information  Module 

BIOL 

Biological 

C30TM 

BIP 

Battlefield  Imaging  Projectile 

BIT 

Built  In  Test 

C4 

BITE 

Built-In  Test  Equipment 

BL 

Battle  Lab 

BLK  I,  BLK  II 

Various  Fiscal  Year  (FY)  Dates 

That  Indicate  When  A  Product 

Should  Be  Ready  (For  Example: 

C4I 

BLK  1  2010  -  TRL6  By  03  BLK 

1) 

Beyond  Line  Of  Sight 

C4ISR 

BLOS 

BLRSI 

Battle  Lab  Reconfigurable 

Simulator  Initiative 

CA 

BLSE 

Battle  Lab  Support  Element 

CA 

BLT 

Battalion  Landing  Team 

CAA 

BLUFOR 

Blue  Force 

CAAM 

BLWE 

Battle  Lab  Warfighting 

Experiment 

CAAV 

BMDO 

Ballistic  Missile  Defense 

Organization 

CAB 

BMO 

Battalion  Maintenance  Officer 

CAD 

BN 

Battalion 

CAE 

BOEING 

Designated  Authors  Of  The 

System  Of  Systems  Architecture 

CAL 

(SOSA) 

CAOC 

BOIP 

Basis-Of-Issue  Plan 

CAP 

BOS 

Battlefield  Operating  Systems 

BP 

Battle  Position 

CAPCES 

BR 

Branch;  British 

BRAC 

Base  Realignment  And  Closure 

BRAG 

Brigade  Artillery  Group 

CAPS 

BRDM 

Russian-Built  Four-Wheeled 

CAR 

Armored  Car,  Used  For  Recon 

And  Anti-Tank  Rocket  Launcher 

CARD 

BRT 

Brigade  Reconnaissance  Troop 

CAS 

BSA 

Brigade  Support  Area 

CASTFOREM 

BSC 

Battle  Simulation  Center 

BSFV-E 

Bradley  Stinger  Fighting 

CAT 

Vehicle-Enhanced 

CATS 

BT 

Basic  Training 

BTOE 

Base  TOE 

CATTB 

BTR 

Russian  Built  Eight- Wheeled 

CAV 

Armored  Personnel  Carrier 

CBE 

Btry 

Battery  (Basic  Artillery  Unit) 

CBO 

BV-ACTD 

Battlefield  Visualization  ACTD 

Budget  Year 
Command  And  Control 
Command,  Control,  And 
Intelligence 

Command  And  Control  Vehicle 
Command  And  Control  Warfare 
Command,  Control,  And 
Communications 
Command,  Control, 
Communications,  And 
Intelligence 

Command,  Control,  And 
Communications  On  The  Move 
Command,  Control, 
Communications,  And 
Computers 
Command,  Control, 
Communications,  Computers, 
And  Intelligence 
Command,  Control, 
Communications,  Computers, 
Intelligence,  Surveillance,  And 
Reconnaissance 
Civil  Affairs;  Combined  Arms 
Civil  Affairs 

Concepts  Analysis  Agency 
Computer-Assisted  Artillery 
Meteorological 
Company  Autonomous  Air 
Vehicle 

Combat  Aviation  Brigade 
Course  Administrative  Data 
Component  Acquisition 
Executive 
Caliber 

Combined  Air  Operations  Center 
Combat  Air  Patrol;  Crisis-Action 
Planning 

Construction  Appropriation, 
Programming,  Control,  And 
Execution  System 
Counteractive  Protection  System 
Chief,  Army  Reserve 
Concept  Analysis  And 
Requirements  Determination 
Close  Air  Support 
Combined  Arms  And  Support 
Task  Force  Evaluation  Model 
Catalog 

Combined  Arms  Training 
Strategy 

Combined  Arms  Tank  Test  Bed 
Cavalry 

Command  Budget  Estimate 
Congressional  Budget  Office 
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CBR 

Chemical,  Biological  And 

CH;  CHAP 

Cargo  Helicopter  Chaplain 

Radiological 

CHS 

Combat  Health  Support 

CBRS 

Combat-Based  Requirements 

Cl 

C  ounterintelligence 

System 

CIA 

Central  Intelligence  Agency 

CBS 

Corps  Battle  Simulation 

CIC 

Combat  Information  Center 

CBS-X 

Continuing  Balance  System- 

CID 

Combat  Intelligence  Division 

Expanded 

CINC 

Commander-In-Chief 

CBT 

Combat 

CINC’s 

Commanders  -  In  -  Chief 

CBTDEV 

Combat  Developer;  Combat 

CIR 

Critical  Information  Requirement 

Development 

CIS 

Capital  Investment  Strategy 

CBU 

Cluster  Bomb  Unit 

CITV 

Commander’s  Independent 

CC 

Closed  Caption 

Thermal  Viewer 

CCD 

Camouflage,  Concealment,  And 

CJCS 

Chairman,  Joint  Chiefs  Of  Staff 

Deception 

CJCS 

Chairman  Of  The  Joint  Chiefs  Of 

CCH 

Close  Combat  Heavy 

Staff 

CCH 

Chief  Of  Chaplains 

CJCSI 

Chairman  Of  The  Joint  Chiefs  Of 

CCIR 

Commander’s  Critical 

Staff  Instruction 

Information  Requirements 

CJTF 

Commander,  Joint  Task  Force 

CCSS 

Commodity  Command  Standard 

CJTFEX 

Combined  Joint  Task  Force 

System 

Exercise 

CCTT 

Close  Combat  Tactical  Trainer 

C-KEM 

Compact  Kinetic  Energy  Missile 

CD 

Cavalry  Division 

CLL 

Chief  Of  Legislative  Liaison 

CDR 

Commander 

CLS 

Contractor  Logistics  Support 

CDS 

Container  Delivery  System 

CM 

Command  Manager 

CDS 

Congressional  Descriptive 

CM(FS) 

Command  Manager  (Force 

Summaries 

Structure) 

CE 

Command  Element 

CM(PBG) 

Command  Manager  (Program 

CE 

Chemical  Energy 

Budget  Guidance) 

CECOM 

Communications  &  Electronics 

CMC 

Civil-Military  Cell 

Command  (U.S.  Army) 

CMCC 

Corps  Movement  Control  Center 

CED 

Concept  Exploration  Design 

CMD 

Command 

CEM 

Common  Engagement  Module 

CMISE 

Corps  Military 

CENTAF 

Central  Command  Air  Force 

Intelligencesupport  Element 

GENICOM 

Central  Command 

CML 

Chemical 

CEOI 

Communications-Electronics 

CMMC 

Corps  Materiel  Management 

CEP 

Concept  Evaluation  Program 

Center 

CEP 

(1)  Circular  Error  Probable;  (2) 

CMO 

Civil-Military  Operations 

Concept  Exploration  Program 

CMOC 

Civil-Military  Operations  Center 

CEP 

Circular  Error  Probable 

CMTC 

Combat  Maneuver  Training 

CEP 

Concept  Evaluation  Proposal 

Center 

CEPCSS 

Centralized  Equipment 

CNGB 

Chief,  NGB 

Procurement  Conversion 

CNR 

Combat  Net  Radio 

Capability  System  Operating 

CO 

Company 

Instructions 

CO 

Commanding  Officer 

CEWI 

Combat  Electronic  Warfare 

COA 

Course  Of  Action 

Intelligence 

COB 

Carrier  Onboard 

CFA 

Covering  Force  Area 

COC 

Council  Of  Colonels 

CFC 

Chairman  Fiscally  Constrained 

COCOM 

Combatant  Command 

CFST 

Critical  Fire  Support  Task 

COE 

Command  Operating 

CFT 

Captive  Flight  Test 

Environment 

CG 

Commanding  General, 

COE 

Common  Operating  Environment 

Chairman’s  Guidance 

COE 

Contemporary  Operational 

CGS 

Common  Ground  Station 

Environment 

CGSC 

Command  Group;  Commanding 

COE 

Corps  Of  Engineers;  Chief  Of 

General  U.S.  Army  Command 

And  General  Staff  College 

Engineers 
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COEA 

Cost  And  Operational 

CRS 

Chairman’s  Readiness  System 

Effectiveness  Analysis 

CS 

Combat  Support 

COI 

Critical  Operational  Issue 

CSA 

Chief  Of  Staff,  U.S.  Army 

COLT 

Combat  Observation  &  Lasing 

CSAR 

Combat  Search  And  Rescue 

Team 

CSB 

Corps  Support  Battalion 

COS 

Chief  Of  Staff 

CSC 

Combat  Stress  Control 

COMARFOR 

Commander,  ARFOR 

CSE 

Combat  Support  Equipment 

COMDT 

Commandant 

CSG 

Corps  Support  Group 

COMINT 

Communications  Intelligence 

CSH 

Combat  Support  Hospital 

Commo 

Communications,  May  Be  Of 

CSM 

Command  Sergeant  Major 

Any  Type;  Voice,  Digital,  Etc. 

CSMA 

Carrier  Sense  Multiple  Access 

Common  Missile 

One  Missile  For  Many  Different 

CSP 

Cost  Schedule  Performance 

Vehicles-Tri-Mode  Warhead. 

CSS 

Combat  Service  Support 

COMMZ 

Communications  Zone 

CSSCS 

Combat  Service  Support  Control 

COMPO 

Component 

System 

COMPT 

Comptroller 

CSSE 

Combat  Service  Support  Element 

COMSAT 

Communications  Satellite 

CT 

Customer  Test 

COMSEC 

Communications  Security 

CTA 

Cased  Telescoped  Ammunition 

CON 

Control 

CTA 

Common  Table  Of  Allowances 

CONPLAN 

Concept  Plan;  Contingency  Plan 

CTAPS 

Contingency  Theater  Automated 

CONUS 

Continental  United  States 

Planning  System 

CONUSA 

Continental  United  States  Army 

CTC 

Combat  Training  Center 

CONUSA 

Continental  U.S.  Army 

CTCP 

Combat  Trains  Command  Post 

COP 

Common  Operating  Picture 

CTD 

Concept  And  Technology 

CORDS 

Civil  Operations  Revolutionary 

Demonstration 

Development  Support 

CTIL 

Commander’s  Tracked  Item  List 

COSCOM 

Corps  Support  Command 

CTOC 

Corps  Tactical  Operations  Center 

COTS 

Commercial-Off-The-Shelf 

CTSF 

Central  Technical  Support 

CP 

Command  Post;  Computer 

Facility 

Program 

CTU 

Consolidated  TOE  Update 

CP 

Command  Post 

CV 

Commander’s  Vehicle 

CP 

Depending  On  Usage,  Command 

CVC 

Combat  Vehicle  Communication 

Post  Or  Check  Point 

CWRP 

Chemical  Warfare  Request 

CPA 

Chairman’s  Program 

Procedures 

Assessment;  Chief  Of  Public 

CX 

Categorical  Exclusion 

Affairs 

CY 

Current  Y ear 

CPG 

Contingency  Planning  Guidance 

D&O 

Doctrine  And  Operational 

CPLAN 

Command  Plan 

D&SA 

Depth  &  Simultaneous  Attack 

CPM 

Critical  Path  Method 

D3A 

Decide,  Detect,  Deliver  &  Assess 

CPR 

Chairman’s  Program 

DA 

Department  Agriculture 

Recommendation 

DA 

(1)  Department  Of  The  Army; 

CPSE 

Corps  Psyop  Support  Element 

(2)  Decision  Aid 

CPT 

Captain 

DAAG 

Data-At-A-Glance 

CPU 

Central  Processing  Unit 

DAB 

Defense  Acquisition  Board 

CPX 

Command  Post  Exercise 

DAB 

Director  Of  The  Army  Budget 

CRA 

Continuing  Resolution  Authority 

(Used  To  Refer  To  The  Deputy 

CRAMM 

Complete  Responsive  Accurate 

Assistant  Secretary  Of  The  Army 

Mission  Module-Manned 

For  Budget)  Defense  Acquisition 

CRC 

Control  And  Reporting  Center 

Board 

CROP 

Command  Research  Objectives 

DAE 

Department  (Of  Defense) 

Plan;  Common  Relevant 

Acquisition  Executive 

Operating  Picture;  Container 

DAG 

Division  Artillery  Group 

Roll-ln/Roll-Out  Platform 

DAG 

Dynamic  Airspace  Management 

CRP 

Combat  Reconnaissance  Patrol 

System 

CRRC 

Construction  Requirements 

DAG 

Data  Authentication  Group 

Review  Committee 

DALSO 

DA  Logistics  Systems  Officer 
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DAMPL 

DA  Master  Priority  List 

DARI 

Detect,  Acquire,  Recognize,  And 
Identify 

DARNG 

Director  Of  The  Army  National 
Guard 

DARPA 

Defense  Advanced  Research 
Projects  Agency 

DAS 

Director  Of  The  Army  Staff 

DASC 

Department  Of  The  Army  Direct 
Air  Support  Center 

DA-WAM 

Deep-Attack  Wide-Area 
Munitions 

DBSL 

Deep  Battle  Synchronization 

Line 

DBST 

Digital  Battle-Staff  Sustainment 
Trainer 

DCA 

Defensive  Counter  Air 

DCG 

Deputy  Commanding  General 

DCS 

Deputy  Chief  Of  Staff 

DCSCD 

Deputy  Chief  Of  Staff  For 
Combat  Developments, 

TRADOC 

DCSINT 

Deputy  Chief  Of  Staff  For 
Intelligence 

DCSLOG 

DCS  For  Logistics 

DCSOPS 

DCS  For  Operations  And  Plans 

DCSPER 

Deputy  Chief  Of  Staff  For 
Personnel 

DCT 

Digital  Communications 

Terminal 

DCX 

Division  Capstone  Exercise 

DDR&E 

Director,  Defense  Research  And 
Engineering 

DE 

Directed  Energy 

DEC 

December 

DECON 

Decontamination 

DEF 

Defense 

DEH 

Directorate  Of  Engineering  And 
Housing  (Now  Known  As  DPW) 

DEN 

Dental 

DEP 

Deputy 

DEPSECDEF 

Deputy  SECDEF 

DET 

Detachment 

DET 

Displaced  Equipment  Training 

DETP 

Displaced  Equipment  Training 
Plan 

DEUCE 

Deployable  Universal  Combat 
Earthmover 

DEW 

Direct-Energy  Weapons 

DF 

Direction  Finding 

DFAS 

Defense  Finance  And 

Accounting  Service 

DFBS 

Defense  Finance  Battlefield 
System 

DFSCOORD 

Deputy  Fire  Support  Coordinator 

DGCS 

Downsized  Ground  Control 
Station 

DI 

Document  Integrator 

DIA 

Defense  Intelligence  Agency 

DII 

Defense  Information 
Infrastructure 

DIR 

Director 

DIRCM 

Directional  Infrared 
Countermeasure 

DIS 

Distributed  Interactive 

Simulation 

DISA 

Defense  Information  Systems 
Agency 

DISC4 

Director  Of  Information  Systems 
For  Command,  Control, 
Communications,  And 

Computers 

DISC4 

Director  Of  Information  Systems 
For  Command,  Control, 
Communications,  And 

Computers 

DISCOM 

Division  Support  Command 

DISE 

Deployable  Intelligence  Support 
Element 

DISP 

Disposal 

DIV 

Division 

DIV  CAV 

Division  Cavalry  Squadron 

DLA 

Defense  Logistics  Agency 

DM 

Director  Of  Management 

DMA 

Defense  Mapping  Agency 

DMAIN 

Division  Main  Command  Post 

DMD 

Digital  Message  Device 

DME 

Displace,  Move,  Emplace 

DMMF 

Division  Mobile  Maintenance 
Facility 

DOCC 

Deep  Operations  Coordination 
Cell 

DOD 

Department  Of  Defense 

DOD 

Department  Of  Defense 

DODAAC 

Department  Of  Defense  Activity 
Address  Code 

DODD 

DOD  Directive 

DoE 

Department  Of  Energy 

DOS 

Days  Of  Supply;  Department  Of 
State 

DOTLMS 

Doctrine,  Organization,  Training, 
Leader  Development,  Materiel, 
And  Soldiers 

DP 

Dual  Purpose 

DPAE 

Director  Of  Program  Analysis 
And  Evaluation 

DPAMMH 

Direct  Productive  Annual 
Maintenance  Man-Hours 

DPAS 

Defense  Priorities  And 

Allocation  System 
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DPDA 

Defense  Property  Disposal 

EBC 

Enroute  Battle  Command 

Agency 

EBC 

Embedded  Battle  Command 

DPG 

Defense  Planning  Guidance 

EC 

Emerging  Concepts 

DPICM 

Dual-Purpose  Improved 

ECB 

Engineer  Combat  Battalion 

Conventional  Munition 

ECBRS 

Enhanced  Concept  Based 

DPM 

Decision  And  Planning  Module 

Requirements  System 

DPP 

Dedicated  Procurement  Program 

ECM 

Electronic  Countermeasures 

DPRG 

Defense  Planning  And  Resources 

ECP 

Emergency  Command 

Board 

Precedence 

DPW 

Director  Of  Public  Works 

EDAS 

Enlisted  Distribution  And 

DRB 

Division  Ready  Brigade 

Assignment  System 

DRB 

Defense  Resources  Board 

E-date 

Effective  Date 

DRMO 

Defense  Reutilization  And 

EDI 

Electronic  Data  Interchange 

Marketing  Office 

EDSS 

Equipment  Distribution 

DRU 

Dynamic  Reference  Unit 

Sequence  System 

DRVT 

Downsized  Remote  Video 

EEFI 

Essential  Elements  Of  Friendly 

Terminal 

Information 

DS 

Direct  Support 

EELS 

Early  Entry  Lethality  And 

DS  FA 

Direct  Support  Field  Artillery 

Survivability 

(Principal  Supporting  Batteries) 

EFAMS 

Enhanced  Fuel  Armament 

DSA 

Division  Support  Area 

Management  Subsystem 

DSI 

Defense  Simulation  Internet 

EFFECTS 

New  Term  For  Artillery  Support. 

DSMAC 

Digital  Scene  Matching  And 

EFOG-M 

Enhanced  Fiber  Optic  Guided 

Area  Correlation 

Missile 

DT 

Developmental  Testing 

EFP 

Explosively  Formed  Penatrator 

DT&E 

Development  Test  And 

EGI 

Embedded  Global 

Evaluation 

Positioning/Inertial  Navigation 

DTAC 

Division  Tactical  Command  Post 

System 

DTG 

Date-Time  Group 

EIS 

Environmental  Impact  Statement 

DTLOMS 

Doctrine,  Training,  Leader 

ELCT 

Electronics 

Development,  Organizations, 

ELIM 

Enlisted  Loss  Inventory  Model 

Materiel,  And  Soldiers 

ELIM-COMPLIP 

Enlisted  Loss  Inventory  Model- 

DTSS 

Digital  Topographic  Support 

Computation  Of  Manpower 

System 

Using  Linear  Programming 

DTT 

Doctrine  And  Tactics  Training 

ELINT 

Electronic  Intelligence 

DTV 

Driver’s  Thermal  Viewer 

EMC 

Electro-Magnetic  Compatible 

DU 

Depleted  Uranium 

EMFCS 

Enhanced  Mortar  Fire  Control 

DY 

Design  Year 

System 

DZ 

Drop  Zone 

EMG 

Electro-Magnetic  Gun 

E3 

Electromagnetic  Environmental 

EMI 

Eloctromagnetic  Interference 

Effects 

EMI 

Electro-Magnetic  Interference 

E4 

Specialist 

EMP 

Electromagnetic  Pulse 

E5 

Sergeant 

EMV 

Eloctromagnetic  Vulnerability 

EA 

Engagement  Area 

EN 

Engineers,  Both  Letters 

EA 

Environmental  Assessment 

Capitalized. 

EA 

Electronic  Attack;  Engagement 

ENGR 

Engineer 

Area;  Each 

ENSCD 

Enemy  Situation  Correlation 

EAC 

Eastern  Area  Command; 

Division 

Echelons  Above  Corps 

EOD 

Explosive  Ordnance  Disposal 

EAC 

Echelon- Above-Corps 

EOM 

End  Of  Mission 

EAC 

Echelons  Above  Corps; 

EOM 

End  Of  Message 

Evaluation  Analysis  Center 

EOTAS 

Electro-Optical  Target 

EAD 

Echelon- Above-Division 

Acquisition  System 

EAD 

Echelons  Above  Division 

EP 

Electronic  Protection 

EADSIM 

Extended  Air  Defense 

EPDS 

Electronic  Processing  And 

Simulation 

Dissemination  System 
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EPG 

Electronic  Proving  Ground 

FBCB2 

Force  XXI  Battle  Command 

EPLRS 

Enhanced  Position  Location 

Battalion/Brigade  And  Below 

Reporting  System 

FCS 

Future  Combat  System 

EPR 

Environmental  Program 

FCS  LSI 

FCS-Lead  System  Integrator 

Requirement 

FCS-ARCMS 

FCS-Advanced  Robotic 

EPW 

Enemy  Prisoner  Of  War 

Countermine  System 

EQUIP 

Equipment 

FCS-AREMS 

FCS-Advanced  Robotic 

ERA 

Explosive  Reactive  Armor 

Engagement  Mortar  System 

ERC 

Equipment  Readiness  Codes 

FCS-ARERS 

FCS-Advanced  Robotic 

ERC 

Equipment  Readiness  Code 

Engagement  Rocket  System 

ERC/DAMPL 

Equip.  Readiness  Codes  &  The 

FCS-ARES 

FCS-Advanced  Robotic 

DA  Master  Pri.  List 

Engagement  System 

ERGM 

Extended  Range  Guided 

FCS-ARRS 

FCS-Advanced  Robotic 

Munition 

Reconnaissance  System 

ER-MLRS 

Extended  Range  Multiple 

FCS-ARSS 

FCS-Advanced  Robotic 

Launch  Rocket  System 

Sustainment  System 

ERPS 

Equipment  Release  Priority 

FCS-ARTAS 

FCS-Advance  Robotic  Target 

System 

Acquisition  System 

ES 

Electronic  Warfare  Support 

FCS-CBT 

FCS-Combat 

ES 

End  Strength 

FCS-FC2V 

FCS-Future  Command  And 

ESL 

Enhanced  Single  Laser 

Control  Vehicle  (System) 

ET 

Embedded  Training 

FCS-FCDR 

FCS-Future  Commander’s 

ETC 

Electro-Thermal  Chemical 

(System) 

EUCOM 

U.S.  Army  European  Command 

FCS-FICV 

FCS-Future  Infantry  Carrier 

EVAC 

Evacuation 

Vehicle  (And  Light  (L)) 

EVD 

Early- Vers  ion  Demonstration 

FCS-FIFV 

FCS-Future  Infantry  Fighting 

EW 

Electronic  Warfare 

Vehicle 

EWO 

Electronic  Warfare  Officer 

FCS-FMV 

FCS-Future  Medical  Vehicle 

EXCOM 

Executive  Committee 

(System) 

EXEVAL 

External  Evaluation 

FCS-FRMV 

FCS-Future  Recovery  And 

EXEVAL 

External  Evaluation 

Maintenance  Vehicle  (System) 

EXFOR 

Experimental  Force 

FCS-FRS 

FCS-Future  Resupply  System 

FA 

Field  Artillery 

FCS-FRV 

FCS-Future  Reconnaissance 

FA(R) 

(Indicates  A  Reinforcing  Field 

Vehicle 

Artillery  Unit) 

FCS-FSPH 

FCS-Future  Self-Propelled 

FAA 

Forward  Assembly  Area 

Howitzer 

FAA 

Functional  Area  Assessment 

FCS-FUV 

FCS-Future  Utility  Vehicle 

FA  AD 

Forward  Area  Air  Defense 

FD 

Fire  Direction  Or  Forward 

FAAD-C2I 

Forward  Area  Air  Defense 

Detachment  (Depending  On 

Command,  Control,  And 

Usage) 

Intelligence  System 

FD/SC 

Failure  Definition/Scoring 

FAASV 

Field  Artillery  Ammunition 

Criteria 

Supply  Vehicle 

FDC 

Fire  Direction  Center 

FAAV 

Family  Of  Autonomous  Air 

FDD 

First  Digitized  Division 

Vehicles  (AAV) 

FDD 

Force  Design  Directorate, 

FAB 

Field  Artillery  Brigade 

DCSCD,  TRADOC 

FAC 

Forward  Air  Controller 

FDO 

Fire  Direction  Officer 

FAIO 

Field  Artillery  Intelligence 

FDTE 

Force  Development  Test  And 

Officer 

Experimentation 

FAR 

Federal  Acquisition  Requirement 

FDU 

Force  Design  Update 

FARP 

Forward  Arming  And  Refueling 

FEB 

February 

Point 

FEBA 

Forward  Edge  Of  The  Battle 

FASCAM 

Family  Of  Scatterable  Mines 

Area 

FASTALS 

Force  Analysis  Simulation  Of 

FEBA 

Forward  Edge  Of  Battle  Area 

Theater  Administrative  And 

FED 

Forward  Entry  Device 

FB 

Finance  Battalion 
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FEMA 

Federal  Emergency  Management 

FSC 

Forward  Support  Company 

Agency 

FSCC 

Fire  Support  Coordination  Center 

FFR 

Force  Feasibility  Review 

FSCL 

Fire  Support  Coordination  Line 

FG 

Finance  Group 

FSCM 

Fire  Support  Coordination 

FI 

Force  Integrator 

Measures 

FIA 

Force  Integration  Analysis 

FSCOORD 

Fire  Support  Coordinator 

FID 

Foreign  Internal  Defense 

FSD 

Full  Scale  Development 

FIFA 

Force  Integration  Functional 

FSE 

Fire  Support  Element 

Area 

FSE 

Fire  Support  Element 

FIN 

Finance 

FSNCO 

Fire  Support  Non-Commissioned 

FIST 

Fire  Support  Team 

Officer 

FISTDA 

Full  Time  Support  TDA 

FSO 

Fire  Support  Officer 

FIST-V 

Fire  Support  Team  Vehicle 

FSR 

Field  Service  Regulation 

FLB 

Forward  Logistics  Base 

FSSG 

Force  Service  Support  Group 

FLIR 

Forward-Looking  Infrared 

FTC 

Forward  Test  Center 

FLO 

Fighter  Liaison  Officer 

FTS 

Full-Time  Support 

FLOT 

Forward  Line  Of  Own  Troops 

FTSMC 

Full  Time  Support  Management 

FLS 

Future  Leaders  And  Soldiers 

Center 

FM 

Field  Manual;  Financial 

FTSTDA 

Full  Time  Support  Table  Of 

Management 

Distribution  And  Allowance 

FM;CFF 

Fire  Mission;Call-For-Fire 

FTX 

Field  Training  Exercise 

FMBT 

Future  Main  Battle  Tank 

FUE 

First  Unit  Equipped 

FMF 

Fleet  Marine  Forces 

FUED 

First  Unit  Equipped  Date 

FMS 

Foreign  Military  Sales 

FVC 

Force  Validation  Committee 

FMSP 

Foreign  Military  Sales  Program 

FWAAV 

Fixed-Wing  Autonomous  Air 

FMTI 

Future  Missile  Technology 

Vehicle 

Institution 

FWD 

Forward 

FMTV 

Family  Of  Medium  Tactical 

FY 

Fiscal  Year 

Vehicles 

FYDP 

Future  Years  Defense  Program 

FNSI 

Finding  Of  No  Significant 

G&C 

Guidance  &  Control 

Impact 

G/VLLD 

Ground/V ehicular  Laser  Locator 

FO 

Forward  Observer 

Designator 

FOA 

Field  Operating  Agency 

G1 

Army  Component  Manpower  Or 

FOB 

Forward  Operating  Base 

Personnel  Staff  Officer  (Army 

FOC 

Future  Army  Operational 

Division  Or  Higher  Staff) 

Capability 

G2 

Army  Component  Intelligence 

FOCA 

Future  Operational  Capabilities 

Staff  Officer  (Army  Division  Or 

Assessment 

Higher  Staff) 

FOG-M 

Fiber-Optic  Guided  Missile 

G2 

Division  Level  Intelligence 

FORMDEPS 

Forscom  Mobilization  And 

Office/Officer 

Deployment  Planning  System 

G3 

Division  Level  Plans  & 

FORMS 

Forward  Observer  Ranging  & 

Operations  Office/Officer 

Marking  System 

G3 

Army  Component  Operations 

FORSCOM 

U.S.  Army  Forces  Command 

Staff  Officer  (Army  Division  Or 

FOS 

Forward  Observer  System 

Higher  Staff) 

FOT 

Follow  On  Operational  Test 

G4 

Army  Component  Logistics  Staff 

FOV 

Field  Of  View 

Officer  (Army  Division  Or 

FP 

Firing  Position 

Higher  Staff) 

FPS 

Facility  Planning  System 

G5 

Assistant  Chief  Of  Staff  Civil 

FRAGO 

Fragmentary  Order 

Affairs  (Army  Division  Or 

FS 

(1)  Future  Systems;  (2)  Fire 

Higher  Staff) 

Support 

Gal. 

Gallons 

FS 

Force  Structure 

GAO 

General  Accounting  Office 

FSA 

Force  Structure  Allowance 

GBCS 

Ground-Based  Common  Sensor 

FSAC 

Fire  Support  Armaments  Center 

GBR 

Ground-Based  Radar 

FSB 

Forward  Support  Battalion 
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GCCS 

Global  Command  And  Control 

HIDACZ 

High-Density  Airspace  Control 

System 

Zone 

GCE 

Ground  Combat  Element 

HIMAD 

High-  To  Medium- Altitude  Air 

GCS 

Ground  Control  Station 

Defense 

GDT 

Ground  Data  Terminal 

HIMARS 

High  Mobility  Artillery  Rocket 

GED 

General  Equivalency  Diploma 

System 

GEN 

General 

HIMARS 

High-Mobility  Rocket  System 

GENESIS 

Generic  Smart  Indirect  Fire 

His 

Human  System  Integration 

Simulation 

HIST 

Historian 

GIE 

Global  Information  Environment 

HL-UAV 

Hand-Launched  Unmanned 

GIS 

Geographic  Information  System 

Aerial  Vehicle 

GLO 

Ground  Liaison  Officer 

HMD 

Helmet  Mounted  Display 

GLPS 

Gun-Laying  And  Positioning 

HMEE 

High-Mobility  Excavation 

System 

Equipment 

GMTI 

Ground  Moving-Target  Indicator 

HMMWV 

High  Mobility  Multipurpose 

GO 

General  Officer 

Wheeled  Vehicle 

GOCOM 

General  Officer  Command 

HN 

Host  Nation 

GOSC 

General  Officer  Steering 

HNSC 

House  National  Security 

Committee 

Committee 

GP 

Group 

HOMES 

Housing  Operations  Management 

GPS 

Global  Positioning  System 

System 

GRCS 

Guardrail  Common  Sensor 

HOW;MSN 

Howitzer;Mission 

GS 

General  Support 

HPC 

High  Performance  Computing 

GSA 

General  Services  Administration 

HPCWG 

High  Performance  Computing 

GSM 

Ground  Station  Module 

Working  Group 

GSR 

General  Support  Reinforcing 

HPSS 

Helmet  Position  Sensing  System 

GSR 

Ground  Surveillance  Radar 

HPT 

High-Payoff  T  arget 

GS-R 

General  Support-Reinforcing 

HPTL 

High  Pay-Off  Target  List 

GSTAMIDS 

Ground  Standoff  Mine  Detection 

HQ 

Headquarters 

System 

HQDA 

Headquarters,  Department  Of 

GT 

General  Test 

The  Army 

GY 

Guidance  Year 

HQIFS 

Headquarters  Integrated 

HAC 

House  Appropriations 

Facilities  System 

Committee 

HQISR 

Headquarters  Installation  Status 

HBC 

House  Budget  Committee 

Report 

HE 

High  Explosive 

HQRPLANS 

Headquarters  Real  Property 

HEAT 

High  Explosive  Anti-Tank 

Planning  And  Analysis  System 

HEDP 

High  Explosive  Dual  Purpose 

HR 

Hour 

HEL 

Helicopter 

Hrs 

Hours 

HEMTT 

Heavy  Enhanced  Mobility 

HSIP 

Human  System  Integration  Plan 

Tactical  Truck 

HSV 

Hunter  Surrogate  Vehicle 

HEP 

High  Explosive  Plastic 

HT 

Highway  Traffic 

HEP 

Hybrid  Electric  Propulsion 

HTI 

Horizonatal  Technology 

HEP-T 

High-Explosive  Tracer 

Integration 

HERO 

Hazards  Of  Electromagnetic 

HTTB 

High  Technology  Test  Bed 

Radiation  To  Ordnance 

HUD 

Heads-Up  Display 

HESD 

Helicopter  Electrostatic 

HUD 

Department  Of  Housing  And 

Discharge 

Urban  Development 

HF 

High  Frequency 

HUMINT 

Human  Intelligence 

HFE 

Human  Factors  Engineering 

HVA 

High  Value  Asset 

HG 

Hydrogen  Generator 

HVT 

High-Value  Target 

HH 

Health  Hazard 

HVTL 

High  Value  Target  List 

HHC 

Headquarters  And  Headquarters 

HVY 

Heavy 

Company 

HYEX 

Hydraulic  Excavator 

HHD 

Headquarters  And  Headquarters 

12 

Image  Intensification 

Detachment 

IADS 

Integrated  Air  Defense  System 
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IADT 

Initial  Active  Duty  For  Training 

INC 

Internet  Controller 

IAP 

Initial  Aiming  Point 

INDIV 

Individual 

IAW 

In  Accordance  With 

INF 

Infantry 

IB 

International  Border 

INFO 

Information 

IC3 

Integrated  Combat  Command 

1NG 

Inactive  Army  National  Guard 

And  Control 

INS 

Inertial  Navigation  System 

ICD 

Initial  Concept  Design 

INTEL 

Intelligence 

ICNIA 

Integrated  Communications 

Interoperability 

New  Definition  Includes  Joint, 

Navigation  Identification 

Legacy,  And  Coalition 

Avionics 

Interoperability. 

ICP 

Incremental  Change  Package 

1NVT 

Inventory 

ID 

Infantry  Division 

IOC 

(1)  Initial  Operational  Capability; 

ID 

Identification 

(2)  Industrial  Operations 

ID  (L) 

Infantry  Division  (Light) 

10  T 

Initial  Operational  Test 

ID(M) 

Infantry  Division  (Mechanized) 

IP 

Intervention  Point 

IDA 

Institute  For  Defense  Analysis 

IP 

Initial  Point 

ID  AD 

Internal  Defense  And 

IP 

Internet  Protocol 

Development 

IPA 

Integrated  Program  Assessment 

IDG 

Installation  Design  Guide 

IPB 

Intelligence  Preparation  Of  The 

IDM 

Improved  Data  Modem 

Battlefield 

IDT 

Inactive  Duty  Training 

IPB 

Installation  Planning  Board 

IER 

Information  Exchange 

IPB 

Intelligence  Preparation  Of  The 

Requirement 

Battlefield 

IEW 

Intelligence  Electronic  Warfare 

IPL 

Integrated  Priority  List 

IEW 

Intelligence  &  Electronic 

IPPT 

Integrated  Process  And  Product 

Warfare 

Team 

IEWTD 

Intelligence  Electronic  Warfare 

IPR 

In  Process  Review 

Test  Directorate 

IPS 

Integrated  Program  Summary 

IFCS 

Improved  Fire  Control  System 

IPT 

Integrated  Product/Process  Team 

IFDC 

Improved  Field  Data  Collector 

IR 

Infrared 

IFF 

Identification,  Friend  Or  Foe 

IR&D 

Independent  Research  & 

IFS 

Integrated  Facilities  System 

Development 

IFSAS 

Initial  Fire  Support  Automation 

IRR 

Individual  Ready  Reserve 

System 

IS 

Information  Superiority 

IFV 

Infantry  Fighting  Vehicle 

ISA 

International  Standardization 

IG 

Inspector  General 

Agreements 

I-HMMWV 

Improved  High-Mobility 

ISB 

Intermediate  Staging  Base 

Multipurpose  Wheeled  Vehicle 

ISR 

Intelligence,  Surveillance,  And 

HQ 

Initial  Issue  Quantity 

Reconnaissance 

IKP 

Instructor  And  Key  Personnel 

ISR 

Installation  Status  Report 

ILIR 

Independent  Laboratory  In- 

IT 

Information  Technology 

House  Research 

ITAADS 

Installation  The  Army 

ILMS 

Improved  Launcher  Mechanical 

Authorization  Documents 

System 

System 

ILS 

Integrated  Logistics  Support 

ITOE 

Intermediate  TOE 

ILSP 

Integrated  Logistics  Support  Plan 

ITP 

Individual  Training  Plan 

IMA 

Individual  Mobilization 

ITV 

In-Transit  Visibility 

Augmenters 

IVIS 

Intervehicular  Information 

IMA 

Information  Mission  Area 

System/Radio  Interface  Unit 

IMETP 

International  Military  Education 

J1 

Manpower  And  Personnel 

And  Training  Program 

Directorate  Of  A  Joint  Staff 

IMF 

Intelligent  Minefield 

J2 

Intelligence  Directorate  Of  A 

IMINT 

Imagery  Intelligence 

Joint  Staff 

IMP 

Installation  Master  Plan 

J3 

Operations  Directorate  Of  A 

IMS 

Integrated  Master  Schedule 

Joint  Staff 

IN 

Inch 
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J4 

Logistics  Directorate  Of  A  Joint 

Joint  STARS 

Joint  Surveillance  Target  Attack 

Staff 

Radar  System 

J5 

Plans  Directorate  Of  A  Joint 

JOPES 

Joint  Operations  Planning  And 

Staff 

Execution  System 

J5 

Strategic  Plans  And  Policy 

JP 

Joint  Publication 

Directorate,  The  Joint  Staff 

JPD 

Joint  Planning  Document 

J6 

Command,  Control, 

JPO 

Joint  Petroleum  Office 

Communications,  And  Computer 

JPOC 

Joint  Project  Optic  Cobra 

Systems  Directorate  Of  A  Joint 

JPOTF 

Joint  Psychological  Operations 

Staff 

Task  Force 

J8 

Force  Structure  Resources  And 

JPO-UAV 

Joint  Program  Office  For 

Assessments  Directorate,  The 

Unmanned  Aerial  Vehicles 

Joint  Staff 

JPS 

Joint  Precision  Strikes 

JAAT 

Joint  Air  Attack  Team 

JRB 

JROC  Review  Board 

JANUS 

Joint  Army,  Navy  Uniform 

JRCC 

Joint  Rescue  Coordination 

Simulation 

Center 

JCCC 

Joint  Communications  Control 

JROC 

Joint  Research  Oversight  Council 

Center 

JROC 

Joint  Requirements  Oversight 

JCDB 

Joint  Common  Database 

Council 

JCM 

Joint  Countermine 

JRTC 

Joint  Readiness  Training  Center 

JCMEB 

Joint  Civil-Military  Engineering 

JSCP 

Joint  Strategic  Capabilities  Plan 

Board 

J-SEAD 

Joint  Suppression  Of  Enemy  Air 

JCMOTF 

Joint  Civil-Military  Operations 

Defenses 

Task  Force 

JSOTF 

Joint  Special  Operations  Task 

JCS 

Joint  Chiefs  Of  Staff 

Force 

JCSE 

Joint  Communications  Support 

JSPS 

Joint  Strategic  Planning  System 

Element 

JSR 

Joint  Strategy  Review 

JDEC 

Joint  Documents  Exploitation 

JSTARS 

Joint  Surveillance  Target  Attack 

Center 

Radar  System 

JFACC 

Joint  Force  Air  Component 

JSTARS 

Joint  Surveillance  Target  Attack 

Commander 

Radar  System 

JFC 

Joint  Force  Commander 

JTA-A 

Joint  Technical  Architecture- 

JFLCC 

Joint  Force  Land  Component 

Army 

Commander 

JTCB 

Joint  Targeting  Coordination 

JFUB 

Joint  Facilities  Utilization  Board 

Board 

JIC 

Joint  Intelligence  Center 

JTCG/ME 

Joint  Technical  Coordinating 

JIF 

Joint  Interrogation  Facility 

Group/Munitions  Effectiveness 

JIPC 

Joint  Imagery  Production 

JTDA 

Joint  /  Defense  Table  Of 

Complex 

Distribution  And  Allowance 

JLOTS 

Joint  Logistics  Over  The  Shore 

JTF 

Joint  Task  Force 

JMAO 

Joint  Mortuary  Affairs  Office 

JTIDS 

Joint  Tactical  Information 

JMBPO 

Joint  Military  Blood  Program 

Distriubtion  System 

Office 

JTRS 

Joint  Training  System 

JMC 

Joint  Movement  Center 

JTRS 

Joint  Tactical  Radio  System 

JMCIS 

Joint  Maritime  Command 

JTTP 

Joint  Tactics,  Techniques,  And 

Information  System 

Procedures 

JMEC 

Joint  Materiel  Exploitation 

JVMF 

Joint  Variable  Message  Format 

Center 

JWCA 

Joint  Warfighting  Capabilities 

JMEM 

Joint  Munitions  Effectiveness 

Assessment 

Manual 

JWID 

Joint  Warfighting 

JMFU 

Joint  Meteorological  Forecasting 

Interoperability  Demonstration 

Unit 

KB 

Kilobyte 

JMRO 

Joint  Medical  Regulating  Office 

KE 

Kenetic  Energy 

JMRR 

Joint  Monthly  Readiness  Review 

KIA 

Killed  In  Action 

JOA 

Joint  Operations  Area 

KM 

Kilometer 

JOC 

Joint  Operations  Center 

Km(s) 

Kilometer(S) 
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KPH 

Kilometers  Per  Hour 

LRRDAP 

Long  Range  Research 

KPP 

Key  Performance  Parameter 

Development  And  Acquisition 

KS 

Kansas 

Plan 

KTI 

Key  Technical  Issue 

LRS 

Long-Range  Surveillance 

L/R 

Launch  And  Recovery 

LRSU 

Long-Range  Surveillance  Units 

Labs 

Laboratories 

LRU 

Line  Replaceable  Unit 

LAM-A 

Loitering  Attack  Munitions 

LSI 

Lead  System  Integrator 

LAN 

Local  Area  Network 

LT 

Light 

LAPES 

Low-Altitude  Parachute 

LTHF 

Lighten  The  Heavy  Force 

Extraction  System 

LTOE 

Living  TOE 

LAV 

Light  Armored  Vehicle 

LUT  2 

Limited  User  Test  2 

LBS 

Pounds 

LWR 

Laser  Warning  Receiver 

LC 

Line  Of  Contact 

LWTB 

Land  Warrior  Test  Bed 

LCC 

Land  Component  Commander 

LZ 

Landing  Zone 

LCPK 

Low-Cost  Precision  Kill 

M 

Meter 

LCR 

Light  Cavalry  Regiment 

Ml 

Abrahams  Main  Battle  Tank 

LCSMM 

Life  Cycle  System  Management 

M1A2  SEP 

M1A2  System  Enhancement 

Model 

Program 

LCU 

Lightweight  Computer  Unit 

M2 

Bradley  Fighting  Vehicle  (BFV) 

LCX 

Logistics  Coordination  Exercise 

M3V 

Mobile  Medical  Mentoring 

LD 

Line  Of  Departure 

Vehicle 

LEAP 

Lightweight  Exoatmospheric 

M4  Carbine 

M4  Carbine 

Projectile 

M88 

Heavy  Recover  Vehicle  (US) 

LER 

Loss  Exchange  Ratio 

MAC 

Maintenance  Allocation  Chart 

LGM 

Laser  Guided  Munition 

MACOM 

Major  Army  Command 

LIF 

Logistics  Intelligence  File 

MACV 

Military  Assistance  Command, 

LIM 

Logistic  Information  System 

Vietnam 

LIN 

Line  Item  Number 

MAE 

Mission  Accomplishment 

LINEDIT 

LIN-Edit 

Estimate 

LLDR 

Lightweight  Laser  Designator 

MAG 

Marine  Aircraft  Group 

Rangefinder 

MAGTF 

Marine  Air-Ground  Task  Force 

LLDR 

Lightweight  Laser  Designator 

MAINT 

Maintenance 

Range  Finder 

MAIS 

Mobile  Army  Instrumentation 

LLTR 

Low-Level  Transit  Route 

System 

LMG 

Light  Machinegun 

MAISRC 

Major  Automated  Information 

LNO 

Liaison  Officer 

Systems  Review  Council 

LO 

Liaision  Officer 

MANPRINT 

Manpower  And  Personnel 

LOA 

Limit  Of  Advance 

Integration 

LOG 

Line  Of  Communications 

MANSCEN 

Maneuver  Support  Center 

LOG 

Logistics 

MAPS 

Modular  Azimuth  Positioning 

LOGCAP 

Logistics  Civil  Augmentation 

System 

Program 

Mar 

March 

LOGPAC 

Logistics  Package 

MARC 

Manpower  Requirements  Criteria 

LOGSACS 

Logistics  Structure  And 

MARDIV 

Marine  Division 

Composition  System 

MARFOR 

Marine  Force 

LOM 

Loitering  Attack  Munitions 

MAT 

Materiel 

LOS 

Line  Of  Sight 

MAT  CMD 

Materiel  Command 

LOSAT 

Line-Of-Sight  Antitank 

MATCH 

TAA  Comparison  Report 

LOTS 

Logistics-Over-The-Shore 

MATDEV 

Materiel  Developer;  Materiel 

LP 

Listening  Post 

Development 

LR 

Long  Range 

MAW 

Marine  Aircraft  Wings 

LRC 

Lesser  Regional  Contingency 

MBA 

Main  Battle  Area 

LRC 

Long-Range  Component 

MBI 

Major  Budget  Issue 

LRF 

Laser  Range  Finder 

MBT 

Main  Battle  Tank 

LRIP 

Low  Rate  Initial  Production 

MC 

Mobilization  Component 

284 


MC4 

Medical  Communication  And 

MI-8 

Russian-Built  Heavy  Transport 

Combat  Casualty  Care 

Helicopter 

MCA 

Military  Construction,  Army 

MIBN 

Mechanized  Infantry  Battalion 

MCAGCC 

Marine  Corps  Air  Ground 

MICAD 

Multipurpose  Integrated 

Combat  Center 

Chemical  Agent  Detector 

MCC 

Movement  Control  Center 

MICLIC 

Mine  Clearing  Line  Charge  (A 

MCDM 

Military  Construction,  Defense 

String  Of  Explosives  Pulled  By 

Medical 

A  Rocket  To  Clear  Mines,  Wire, 

MCOO 

Modified  Combined  Obstacle 

Etc.) 

Overlay 

MICOM 

Missile  Command 

MCS 

Maneuver  Control  System 

MICV 

Mechanized  Infantry  Combat 

MCS-P 

Maneuver  Control  System- 

Vehicle 

Phoenix 

MIES 

Modernized  Imagery 

MCT 

Movement  Control  Team 

Exploitation  System 

MDAP 

Major  Defense  Acquisition 

MILCON 

Military  Construction 

Program 

MILDEP 

Military  Deputy 

MDEP 

Management  Decision  Package 

MILES 

Multiple  Integrated  Laser 

MDL 

Mission  Data  Loader 

Engagement  System 

MDM 

Medium 

MIN 

Minute 

MDMP 

Military  Decision  Making 

MIP 

Met  Improvement  Plan 

Process 

MITT 

Mobile  Integrated  Tactical 

MDR 

Milestone  Decision  Review 

Terminal 

MDS 

Meteorological  Data  System 

MLRS 

Multiple-Launch  Rocket  System 

MEB 

Marine  Expeditionary  Brigade 

MM 

Millimeter 

MECH 

Mechanized 

MMC 

Materiel  Management  Center 

MED 

Medical 

MMEWR 

Minimum  Mission  Essential 

MEF 

Marine  Expeditionary  Force 

Wartime  Requirements 

MEGAJULE 

Amount  Of  Energy  Used  In 

MMR 

Multi-Mission  Radar 

EMG  Velocity/Force 

MMS 

Meteorological  Measuring 

Measurements. 

System 

MEL 

Master  Events  List 

MNS 

Mission  Need  Statement 

MELIOS 

Mini-Eye-Safe  Laser  Infrared 

Mnv  Bn  FS 

Maneuver  Battalion  Fire  Support 

Observation  Set 

MOA 

Memorandum  Of  Agreement 

MEMS 

Micro-Electrical  Munitions 

MOBTDA 

Mobilization  TDA 

Systems 

MOC 

Management  Of  Change 

MERM 

Medium-Extended  Range 

MODPATH 

Modernization  Path 

Munitions 

MODSAF 

Modular  Semi-Automated  Force 

Met 

Meteorological 

MOE 

Measure  Of  Effectiveness 

METL 

Mission  Essential  Task  List 

MON 

Memorandum  Of  Notification 

METT-T 

Mission,  Enemy,  Terrain, 

MOP 

Memorandum  Of  Policy 

Troops,  And  Time  Available 

MOPP 

Mission-Oriented  Protection 

MEU 

Marine  Expeditionary  Unit 

Posture 

MFA 

Materiel  Fielding  Agreement 

MOPP-4 

Mission-Oriented  Protection 

MFCS 

Mortar  Fire  Control  System 

Posture  Level  4 

MFDC 

Multifunctional  Data  Collector 

MOS 

Military  Occupational  Specialty 

MFO 

Multinational  Force  Of 

MOSLS 

Military  Occupational  Specialty 

Observers 

Level  System 

MFOM 

MLRS  Family  Of 

MOUT 

Military  Operations  On  Urban 

Munitions/Submunitions 

T  errain 

MFORCE 

Master  Force 

MOV 

Movement 

MFP 

Materiel  Fielding  Plan 

MP 

Military  Police 

MG 

Machine  Gun 

MPDI 

MACOM  POM  Development 

MGB 

Medium  Girder  Bridge 

Instructions 

MGMT 

Management 

MPES 

Mobilization  Planning  And 

MI 

Military  Intelligence 

MPO 

Execution  System 

Mission  Payload  Operator 
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MPU 

Mobile  Power  Unit 

MTSQ 

Mechanical  Time,  Superquick 

MRA 

Manpower  Reserve  Affairs 

MTT 

Mobile  Training  Teams 

MRAAS 

Multi-Role  Armament 

MTTR 

Mean-Time-To-Repair 

Ammunition  System 

MTW 

Major  Theater  War 

MRB 

Motorized  Rifle  Battalion 

MULE 

Modular  Universal  Laser 

MRC 

Major  Regional  Contingency 

Equipment 

MRC 

(1)  Motorized  Rifle  Company; 

MULE 

Wheeled/Tracked  Robotic 

(2)  Major  Regional  Conflict;  (3) 

Transport  Vehicle. 

MRD 

Mechanized  Rifle  Division 

MUN 

Munition 

MRIS 

Modernization  Resource 

MUTA 

Multiple  Unit  Training 

Information  Submission 

Assemblies  (Utas) 

MRL 

Mobile  Rocket  Launcher 

MUTA-4 

Four  Utas  Conducted  Back-To- 

MRL 

Materiel  Requirements  List 

Back  (Normally  One  Weekend 

MRMC 

Medical  Research  And  Materiel 

MUTA) 

Command 

MWR 

Morale,  Welfare,  And  Recreation 

MRR 

Minimum-Risk  Route 

NAF 

Numbered  Air  Force 

MRR 

Motorized  Rifle  Regiment 

NAI 

Named  Area  Of  Interest 

MRSI 

Multiple-Round  Simultaneous 

NATO 

North  Atlantic  Treaty 

Impact 

Organization 

MRSI  (mission) 

Multi-Rounds  Simultaneous 

NAVAID 

Navigational  AID 

Impact 

NBC 

Nuclear,  Biological,  And 

MRT 

Movement  Regulating  Team 

Chemical 

MS 

Microsoft 

NCA 

National  Command  Authorities 

MS 

Milestone 

NCO 

Noncommissioned  Officer 

MS  “B” 

Milestone  "B"  =  April  15,  2003  = 

NDM 

Naturalistic  Decision  Making 

Target  Date  For  FCS  Efforts. 

Model 

MSC 

Military  Sealift  Command; 

NEA 

Northeast  Asia 

Major  Subordinate  Command 

NEO 

Noncombatant  Evacuation 

MSE 

Mobile  Subscriber  Equipment 

Operation 

MSF 

Mobile  Strike  Force 

NEPA 

National  Environmental  Policy 

MSF95 

Mobile  Strike  Force  95 

Act  Of  1969 

MSG 

Multi  Source  Group 

NET 

New  Equipment  Training 

MSIP 

Multispectral  Imagery  Processor 

NET  FIRES 

Intelligence  Integrated  Artillery 

MSL 

Missile 

Fires  System. 

MSP 

Met  Sensing  Package 

NETP 

New  Equipment  Training  Plan 

MSP 

Mission  Support  Plan 

NETT 

New  Equipment  Training  Team 

MSR 

Main  Supply  Route 

NG 

National  Guard 

MSS 

Mission  Support  System 

NG 

Next  Generation 

MSTAR 

MLRS  Smart  Tactical  Rocket 

NGB 

National  Guard  Bureau 

MSTAR 

A  New  Form  Of  Precision 

NGF 

Naval  Gunfire 

Deliverable  Munitions. 

NGFS 

Naval  Gunfire  Fire  Support 

MT 

Mechanical  Time 

NGLO 

Naval  Gunfire  Liaison  Officer 

MTBSA 

Mean-Time -Between-System- 

NGO 

Nongovernment  Organization 

Abort 

NICP 

National  Inventory  Control  Point 

MTF 

Medical  Treatment  Facility 

NITF 

National  Imagery  Transmission 

MTI 

Moving-Target  Indicator 

Format 

MTLD 

Man-Portable  Target  Locating 

NLO 

Naval  Liaison  Officer 

Device 

NLOS 

Non-Line  Of  Sight 

MTMC 

U.S.  Army  Military  Traffic 

NLT 

Not  Later  Than 

Management  Command 

NM 

Nautical  Mile 

MTOE 

Modification  TOE 

NMCM 

Not  Mission  Capable  For 

MTOE 

Modification  Table  Of 

Maintenance 

Organization  And  Equipment 

NMP 

National  Maintenance  Point 

MTP 

Materiel  Transfer  Plan 

NMS 

National  Military  Strategy 

Mtr 

Mortar 

NMSD 

National  Military  Strategy 

MTS 

Marine  Tactical  System 

Document 
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NODE 

Any  Place  Where  Encryption 

OFC 

Office 

And  Decryption  Occurs. 

OFCD 

Objective  Force  Concept  Design 

NOE 

Nap  Of  The  Earth 

OFF 

Officer 

NOF 

Notional  Force 

OFT 

Operational  Feasibility  Test 

NOI 

Notice  Of  Intent 

OH 

Observation  Helicopter 

NOT 

New  Organization  Training 

01 

Organization  Integrator; 

NOV 

November 

Organizational  Integration 

NRC 

National  Research  Council 

OICW 

Objective  Individual  Combat 

NRT 

Near-Real  Time 

Weapon 

NSC 

National  Security  Council 

OJCS 

Office  Of  The  Joint  Chiefs  Of 

NSCS 

National  Security  Council 

Staff 

System 

OLOS 

Organic  LOS 

NSD 

National  Security  Directive 

OMA 

Operation  And  Maintenance, 

NSFS 

Naval  Surface  Fire  Support 

Army 

NSG 

North  Seeking  Gyroscope 

OMB 

Office  Of  Management  And 

NSR 

National  Security  Review 

Budget 

NSS 

Node  Switching  Sites 

OMFTS 

Operational  Maneuver  From  The 

NSTO 

New  System  Training  Office 

Sea 

NTC 

National  Training  Center 

OMS/MP 

Operational  Mode 

NTDR 

Near-Term  Digital  Radio 

Summary/Mission  Profile 

NV 

Night  Vision 

OOD 

Out-Of-DAMPL  Sequence 

NVPS 

Night  Vision  Pilotage  System 

OODA 

Observe,  Orient,  Decide,  Act 

NWC 

Naval  Weapons  Center 

OOTW 

Operations  Other  Than  War 

O&I 

Operations  And  Intelligence 

OP 

Observation  Post 

O&O 

Organizational  And  Operational 

OPALS 

Officer  Projection  Aggregrate 

O&S 

Operations  And  Support 

Level  System 

O/C 

Observer/Controller 

OPALS 

Officer  Projection  Aggregate 

OA 

Operational  Architecture 

Level  System 

OA 

Obligation  Authority 

OPCOM 

Operational  Command 

OAS 

Organization  Of  American  States 

OPCON 

Operational  Control 

OAV 

Organic  Air  Vehicle 

OPFAC 

Operational  Facilities 

OBJ 

Objective 

OPFOR 

Opposing  Force 

OC 

Observer-Controller 

OPLAN 

Operation  Plan 

OCA 

Offensive  Counter  Air 

OPLANS 

Operations  Plans 

OCAR 

Office  Of  The  Chief,  Army 

OPM 

Office  Of  Personnel 

Reserve 

Management 

OCD 

Operational  Concept 

OPORD 

Operation  Order 

Demonstration 

OPS 

Operations 

OCONUS 

Outside  The  Continental  United 

OPSEC 

Operations  Security 

States 

OPTEC 

Operational  Test  &  Evaluation 

OCR 

Operational  Capability 

Command 

Requirement 

OPTEMPO 

Operating  Tempo 

OCSW 

Objective  Crew-Served  Weapon; 

ORD 

Operational  Requirements 

Objective  Crew-Served  Weapon 

Document 

TRL4 

ORF 

Operational  Readiness  Float 

ODCSLOG 

Office  Of  The  Deputy  Chief  Of 

ORSA 

Operations  Research  And 

Staff,  Logistics 

Systems  Analysts 

ODCSOPS 

Office  Of  The  Deputy  Chief  Of 

OS 

Operating  Strength 

Staff,  Operations  And  Plans 

OSA 

Office  Of  The  Secretary  Of  The 

ODCSPER 

Office  Of  The  Deputy  Chief  Of 

Army 

Staff,  Personnel 

OSD 

Office  Of  The  Secretary  Of 

ODH  VI 

Operation  Desert  Hammer  VI 

Defense 

OEC 

Operational  Evaluation 

OSDe 

Operating  Strength  Deviation 

Command 

OSUT 

One  Station  Unit  Training 

OER 

Officer  Efficiency  Report 

OT 

Operational  Testing 

OF 

Objective  Force 

OT 

Operating  Time 
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OT&E 

Operational  Testing  And 

PHTK 

Precision  Hit  To  Kill 

Evaluation 

PIP 

Product  Improvement  Program 

OTC 

Operational  Test  Command 

PIR 

Priority  Information 

OTM 

On-Line  Training  Module 

Requirements 

OTM 

On  The  Move 

PKO 

Peace-Keeping  Operations 

OTN 

Own-The-Night 

PL 

Phase  Line 

OTOE 

Objective  TOE 

PLC 

Pulsed  Logistic  Concept 

OTRR 

Operational  Test  Readiness 

PLGR 

Precision  Lightweight  Global 

Review 

Positioning  System  Receiver 

OTT 

October 

PLL 

Prescribed  Load  List 

OV 

Orbiting  Vehicle 

PLS 

Palletized  Loading  System 

P3I 

Preplanned  Product  Improvement 

PLT 

Platoon 

PA 

Public  Affairs 

PM 

Provost  Marshal 

PA 

Precision  Attack 

PM 

(1)  Project  Manager;  (2)  Program 

PAC-3 

Patriot  Advanced  Capability 

Manager 

Level-3 

PMAD 

Personnel  Management 

PACOM 

US  Army  Pacific  Command 

Authorization  Document 

PADS 

Position  And  Azimuth 

PMC 

Personnel  Management  Center 

Determining  System 

PMCS 

Preventive  Maintenance  Checks 

PAED 

Program  Analysis  And 

And  Services 

Evaluation  Directorate 

PMJ 

Professional  Military  Judgment 

PAM 

Pamphlet 

PO 

Project  Office 

PAM 

Precision  Attack  Munitions 

POC 

Platoon  Operations  Center 

PAO 

Public  Affairs  Officer 

POC 

Point  Of  Contact 

PB 

President’s  Budget 

POD 

Port  Of  Debarkation 

PBAC 

Program  Budget  Advisory 

POE 

Port  Of  Embarkation 

Committee 

POI 

Program  Of  Instruction 

PBC 

Program  Budget  Committee  Of 

POL 

Petroleum,  Oils,  And  Lubricants 

PPBES 

POM 

Program  Objective  Memorandum 

PBC 

Program  And  Budget  Committee 

POS/NAV 

Position  Navigation 

PBD 

Program/Budget  Decision 

POSC-edit 

Personnel  Occupational  Specialty 

PBG 

Program  And  Budget  Guidance 

Code-Edit  File 

PBG 

Program  Budget  Guidance 

POSNAV 

Position  Navigation 

PCS 

Permanent  Change  Of  Station 

POTF 

Psychological  Operations  Task 

PD 

Point  Detonating 

Force 

PDD 

Presidential  Decision  Directive 

PP&O 

Plans,  Programs,  And  Operations 

PDM 

Program  Decision  Memorandum 

PPAR 

Purpose,  Priority,  Allocation  And 

PE 

Program  Element 

Restrictions 

PEG 

Program  Evaluation  Group 

PPBERS 

Program  Perf.  And  Budget  Exec. 

PEGS 

Program  Evaluation  Groups  Of 

Rev.  Sys. 

PPBES 

PPBES 

Planning,  Programming, 

PEO 

Peace  Enforcement  Operations 

Budgeting,  And  Execution 

PEO 

Program  Executive  Officer 

System 

PERS 

Personnel 

PPBS 

Planning,  Programming,  And 

PERSACS 

Personnel  Structure  And 

Budgeting  System 

Composition  System 

PQT 

Production  Qualification  Test 

PERSCOM 

Total  Army  Personnel  Command 

PR 

Personnel  Readiness 

PERSO 

Personnel  Systems  Staff  Officer 

PRB 

Program  Review  Board 

PERT 

Program  Evaluation  And  Review 

PRD 

Presidential  Review  Decision 

Technique 

PREPO 

Prepositioned  Sets  Of  Equipment 

PESD 

Personnel  Electrostatic  Discharge 

PRG 

Program  Review  Group 

PETRI 

Petroleum 

PRGM 

Program 

PG 

Personnel  Group 

PRMD 

Personnel  Readiness 

PGCS 

Portable  Ground  Control  Station 

Management  Division 

PGMM 

Precision  Guided  Mortar 

Munitions 

PRO 

Procedures 
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PROBE 

Program  Optimization  And 

RCEM 

Regional  Contingency 

Budget  Evaluation 

Engineering  Manager 

PROC 

Processing 

RCF 

Repair  Cycle  Float 

PROC 

Procurement  Appropriation 

RCP 

Relevant  Common  Picture 

PROP 

Property 

RD(S) 

Round(S) 

PS 

Personnel  Support 

RDA 

Research,  Development,  And 

PSA 

Port  Support  Activity 

Acquisition 

PSB 

Personnel  Service  Battalion 

RDAP 

Research,  Development,  And 

PSG 

Prioritization  Steering  Group  Of 

Acquisition  Plan 

PPBES 

RDCOMP 

Round(S)  Complete 

PSG 

Prioritization  Steering  Group 

RDD 

Required  Delivery  Date 

PSYOP 

Psychological  Operations 

RDD 

Requirements  Documentation 

PSYOPS 

Psychological  Operations 

Directorate,  USAFMSA 

PT 

Physical  Training 

RDEC 

Research  Development  And 

PTS 

Parts 

Engineering  Center 

PY 

Program  Year;  Prior  Year 

RDF 

Radio  Direction  Finding 

PZ 

Pick-Up  Zone 

RDS 

Requirements  Documentation 

QA 

Quality  Assurance 

System 

QAPR 

Quarterly  Army  Performance 

RDTE 

Research,  Development,  Test, 

Review 

And  Evaluation 

QC 

Quality  Control 

REC 

Record  Of  Environmental 

QE 

Quadrant  Elevation 

Consideration 

QF 

Quick  Fire 

RECON 

Reconnaissance 

QFD 

Quality  Function  Deployment 

Redcon 

Readiness  Condition;  1  =  Ready 

QMP 

Qualitative  Management 

To  Move  Now,  2=  Ready  To 

Program 

Move  In  1 5  Minutes,  3  =  Ready 

QQPRI 

Quan.  &  Qual.  Personnel  Reqr 

To  Move  In  60  Minutes,  4  = 

Info. 

Ready  To  Move  In  One  Hour 

QQPRI 

Qualitative  And  Quantitative 

REFORGER 

Reinforce  Germany 

Personnel  Requirements 

REGT 

Regiment 

Information 

REP 

Representative 

QRMP 

Quick-Response  Multicolor 

REPLMT 

Replacement 

Printer 

REQUEST 

Recruit  Quota  System 

QSTAG 

Quadripartite  Standardization 

REQVAL 

Requisition  Validation 

Agreement 

RETAIN 

Reenlistment,  Reclassification 

QuickLook 

105mm  Tube-Launched  UAV. 

And  Assignment  System 

R 

Reinforcing 

RFL 

Restricted  Fire  Line 

R&D 

Research  And  Development 

RFP 

Request  For  Proposal 

R&M 

Reliability  And  Maintainability 

RFPB 

Reserve  Forces  Policy  Board 

R&S 

Reconnaissance  And 

RFPI 

Rapid  Force  Projection  Initiative 

Surveillance 

RGR 

Ranger 

RAD 

Requirements  Authorization 

RHU 

Replacement  Holding  Unit 

Docuement 

RIMS 

Research  And  Development 

RAM 

Reliability,  Availability,  And 

Information  Management  System 

Maintainability 

RISTA 

Reconnaissance,  Intelligence, 

RAM2000 

Reliability,  Availability,  And 

Surveillance,  And  Target 

Maintainability  2000 

Acquisition 

RAMM 

Responsive  Accurate  Mission 

RLO 

Reconnaissance  Liaison  Officer 

Module 

RLT 

Regimental  Landing  Team 

RAOC 

Rear  Area  Operations  Center 

RMS 

Risk  Management  Strategy 

RAP 

Rocket- Assisted  Projectile 

RMU 

Resource  Management  Update 

RAP 

Revised  Approved  Program 

ROA 

Restricted  Operations  Area 

RC 

Reserve  Component 

ROC-V 

Robotic  Countermine  Vehicle 

RCCC 

Reserve  Component 

ROD 

Record  Of  Decision 

Coordination  Council 

ROE 

Rules  Of  Engagement 

ROM 

Refuel  On  The  Move 
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ROTC 

Reserve  Officers’  Training  Corps 

ROZ 

Restricted  Operations  Zone 

RP 

Release  Point 

RPD 

Recognition  Primed  Decision- 
Making  Model 

RPI 

Real  Property  Inventory 

RPIP 

Real  Property  Investment  Plan 

RPLANS 

Real  Property  Planning  And 
Analysis  System 

RPM 

Real  Property  Maintenance 

RPMA 

Real  Property  Maintenance 
Activities 

RPMP 

Real  Property  Master  Plan 

RPMS 

Real  Property  Management 
System 

RPPB 

Real  Property  Planning  Board 

RPV 

Remotely  Piloted  Vehicle 

RPWS 

Remote  Player  Workstation 

RSC 

Army  Reserve  Regional  Support 
Command 

RSOI 

Reception,  Staging,  Onward- 
Movement,  And  Integration 

RSOP 

Readiness  Standing  Operating 
Procedures 

RSS 

Reconnaissance  Surveillance 

And  Security 

RSTA 

Reconnaissance,  Surveillance, 
And  Target  Acquisition 

RSV 

Resupply  Vehicle 

RTCA 

Real-Time  Casualty  Assessment 

RTO 

Radio-Telephone  Operator 

RUDE 

Receive,  Understand, 
Disseminate,  Execute 

RVT 

Remote  Video  Terminal 

RWAAV 

Rotary  Wing  Autonomous  Air 
Vehicle 

RWS 

Remote  Workstation 

S&I 

Science  And  Infrastructure 

S&T 

Science  And  Technology 

SI 

Battalion  Or  Brigade  Personnel 
Officer 

S2 

Battalion  Or  Brigade  Intelligence 
Officer 

S3 

Battalion  Or  Brigade  Operations 
And  Staff  Officer 

S4 

Battalion  Or  Brigade  Logistics 
Officer 

SA 

Situational  Awareness 

SA 

Secretary  Of  The  Army 

SAC 

Senate  Appropriations 

Committee 

SACC 

Supporting  Arms  Coordination 
Center 

SACS 

Structure  And  Composition 
System 

SAD ARM 

Sense  And  Destroy  Armor 

SADM 

System  Acquisition  Decision 
Memorandum 

SAF 

Semi- Automated  Forces 

SAG 

Senior  Advisory  Group;  Study 
Advisory  Group 

SALUTE 

Size,  Activity,  Location,  Unit, 
Time,  Equipment  (Message) 

SAM 

Surface-To-Air  Missile 

SAMAS 

Structure  And  Manpower 
Allocation  System 

SAPO 

Subarea  Petroleum  Office 

SAR 

Synthetic  Aperture  Radar 

SARDA 

Office  Of  The  Assistant 

Secretary  Of  The  Army 
(Research,  Development,  and 
Analysis) 

SARDB 

Survivable  Armed 
Reconnaissance  On  The  Digital 
Battlefield 

SASC 

Senate  Armed  Services 
Committee 

SASO 

Stability  And  Support  Operations 

SAT 

Systems  Approach  To  Training 

SATP 

Security  Assistance  Training 
Program 

SAW 

Squad  Automatic  Weapon 

SBC 

Senate  Budget  Committee 

SBF 

Support  By  Fire 

SBIR 

Small  Business  Innovation 
Research  Program 

SCIPS 

Standardized  Integrated 
Command  Post  System 

SCP 

Survey  Control  Point 

Set 

Scout 

SDD 

System  Design  Demonstration 

SDD 

System  Development  Definition 

SEAD 

Suppression  Of  Enemy  Air 
Defenses 

SEC 

Section 

SECDEF 

Secretary  Of  Defense 

SELCOM 

Select  Committee  Of  PPBES 

SEMA 

Special  Electronics  Mission 
Aircraft 

SEP 

September 

SEP 

System  Evaluation  Plan 

SERV 

Service 

SESIL 

System  Electronics  And 

Software  Integration  Laboratory 

SF 

Special  Forces 

SF 

Sustained  Fire 

SFC 

Sergeant  First  Class 

SFM 

Sensor  Fused  Munitions 

SFOA 

Special  Forces  Operational  Area 

SFOB 

Special  Forces  Operational  Base 

SFOD 

Special  Forces  Operational 
Detachment 
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SGA 

Standards  Of  Grade 

SOV 

Staff  Operations  Vehicle 

Authorization 

SOW 

Statement  Of  Work 

SGS 

Secretary  Of  The  General  Staff 

SP 

Self-Propelled 

SHOT 

Round(S)  Fired 

SP 

(1)  Self-Propelled;  (2)  Solid 

SHSM 

Soldier  Health  Status  Monitor 

Propellent;  (3)  Single  Purpose 

SI 

Systems  Integrator 

SP 

Start  Point 

SIB 

Separate  Infantry  Brigade 

SPC 

Strategy  Planning  Committee  Of 

SIDPERS 

Standard  Installation  Division 

PPBES 

Personnel  System 

SPCE 

Survey  Planning  And 

SIG 

Signal 

Coordination  Element 

SIGINT 

Signal  Intelligence 

SPH 

Self-Propelled  Howitzer 

SIGINT 

Signals  Intelligence 

SPLASH 

Round(S)  Impacting  In  About  5 

SIMOS 

Space  Imbalanced  MOS 

Seconds 

S1NCGARS 

Single  Channel  Ground  And 

SPLL 

Self-Propelled  Launcher-Loader 

Airborne  Radio  System 

SPNV 

Special  Project  Night  Vector 

SIO’s 

Missions  Of  Standard 

SPT 

Support 

Installation  Organizations 

SQD 

Squad 

SIPC 

Stationing  And  Installation 

SQDN 

Squadron 

Planning  Committee 

SR 

Short  Range 

SIRDAP 

Science  And  Infrastructure  RDA 

SRC 

Standard  Requirement  Code; 

Plan 

Short-Range  Component 

SISA 

Science  And  Infrastructure 

SRP 

Soldier  Readiness  Processing 

Support  Analysis 

ss 

System  Safety 

Sit  Rpt,  Sitrep 

Situation  Report 

ssc 

Small-Scale  Contingencies 

SJA 

Staff  Judge  Advocate 

SSM 

Surface-To-Surface  Missile 

SKA 

Skills,  Knowledge,  And 

SSP 

System  Support  Package 

Attributes 

SSv 

Soldier  Survivability 

SLAM 

Standoff  Land  Attack  Missile 

ST 

Standby  Time 

SLID 

System-Level  Integration 

ST 

Sustainment  Training 

Demonstration 

ST ACCS 

Standard  Theater  Army 

SMART  Mines 

Self-Healing  Mines  That  Will 

Command  And  Control  System 

Follow  An  Opponent. 

STAFF 

Smart  Target  Activated  Fire  And 

SMDR 

Structure  &  Manning  Decision 

Forget 

Review 

STANAG 

Standardized  North  Atlantic 

SME 

Subject  Matter  Experts 

Treaty  Organization  Agreement 

SMI 

Soldier-Machine  Interface 

STANAG 

Standardization  Agreement 

SO 

Special  Operations 

STAR 

System  Threat  Assessment 

SOC 

Special  Operations  Command 

Report 

SOCCE 

Special  Operations  Command 

STARC 

State  Area  Command 

And  Control  Element 

STAW 

Smart  Top  Attack  Weapon 

SOCCH 

Special  Operations  Command 

STD 

System  Technology 

And  Control  Headquarters 

Demonstration 

SOCOM 

Special  Operations  Command 

STO 

Science  &  Technology 

SOCOORD 

Special  Operations  Coordinator 

Objectives 

SOE 

Statement  Of  Equipment 

STO 

Science  And  Technology 

SOF 

Special  Operations  Forces 

Objective 

SOFA 

Status  Of  Forces  Agreement 

STOL 

Short  Take-Off  And  Landing 

SOI 

Signal  Operations  Instructions 

STORM 

Simulation,  Testing,  Operation, 

SOO 

Statement  Of  Objectives 

Rehearsal  Model 

SOP 

Standing  Operating  Procedures 

STOW 

Synthetic  Theater  Of  War 

SORC 

System’s  Operations 

STP 

System  Training  Plan 

Requirements  And  Capabilities 

STP’s 

Short  Term  Projects 

SORTS 

Status  Of  Resources  And 

STRAC 

Standards  In  Weapons  Training 

Training  System 

STRAP 

System  Training  Plan 

SOS 

Speed  Of  Service 

STRICOM 

Simulation,  Training,  And 

SOSA 

System  Of  Systems  Architecture 

Instrumentation  Command 
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STRIKWARN 

Strike  Warning 

TAG 

The  Adjutant  General  Of  A  State 

STTE 

Special  Tools  And  Test 

Or  Territory 

Equipment 

TAI 

Target  Areas  Of  Interest 

STTR 

Small  Business  Technology 

TAIS 

Tactical  Airspace  Integration 

Transfer  Pilot  Program 

System 

STX 

Situational  Training  Exercise 

TALDT 

Total  Administrative  And 

SU 

Situational  Understanding 

Logistics  Delay  Time 

S-UAV 

Shot  Unattended  Vehicle  (Tube- 

TALO 

Tactical  Airlift  Liaison  Ofllcer 

Launched  105mm) 

TALS 

Tactical  Automatic  Landing 

SUBS 

Subsistence 

System 

SUP 

Supply 

TAP 

The  Army  Plan 

SURG 

Surgeon 

TAPDB 

Total  Army  Personnel  Database 

SVC 

Service 

TAPDB-AE 

Total  Army  Personnel  Data 

SVP’s 

Special  Visibility  Programs 

Base-Active  Enlisted 

SWA 

Southwest  Asia 

TAPDB -AE 

TAPDB-Active  Enlisted 

SWG 

Seminar  Wargame  (For  This 

TAPDB -AO 

Total  Army  Personnel  Data 

Iteration  Caspian  Sea  Scenario) 

Base-Active  Officer 

SWO 

Staff  Weather  Officer 

TAPDB-AO 

TAPDB-Active  Officer 

SWOE 

Smart  Weapons  Operability 

TAR 

Tactical  Air  Reconnaissance 

Enhancement 

T ARABS 

Tactical  Air  Reconnaissance  And 

SYNJAM 

Synthetic  Jammers 

Aerial  Battlefield  Surveillance 

SYS 

Systems 

TASOSC 

Theater  Army  Special  Operations 

T&E 

Testing  And  Evaluation 

Support  Command 

T&E  IPT 

Test  And  Evaluation  Integrated 

TAV 

Total  Asset  Visibility 

Process  Team 

TBEP 

Training  Base  Expansion  Plan 

T-80 

Russian  Built  Tank 

TBM 

Tactical  Ballistic  Missile 

TA 

Theater  Army 

TBMD 

Tactical  Ballistic  Missile 

TA 

Target  Acquisition 

Defense 

TAA 

Tactical  Assembly  Area 

TCACCIS 

Transportation  Coordinator 

TAA 

Total  Army  Analysis 

Automated  Command  And 

TAADS 

The  Army  Authorization 

Control  Information  System 

Documents  System 

TCC 

Test  Control  Center 

TAADS-R 

The  Army  Authorization 

TCF 

Tactical  Combat  Force 

Document  SystcnfA  Re-Design 

TCM 

Total  Corrective  Maintenance 

TAC 

Tactical 

TCP 

Traffic  Control  Plan 

TAC 

Tactical  Command  Post 

TCP 

Transmission  Control  Protocol 

TACAIR 

Tactical  Air 

TDA 

Table  Of  Distribution  And 

TACC 

Tactical  Air  Control  Center 

Allowances 

TACFIRE 

Tactical  Fire 

TDA 

Target  Damage  Assessment 

TACFIRE 

Tactical  Fire  Direction  System 

TDA 

Table  Of  Distribution  And 

TACMS 

Tactical  Missile  System  (Navy) 

Allowances 

TACOM 

Tactical  Command 

TDR 

Test  Data  Report 

TACOM 

Tank- Automotive  &  Armaments 

TE 

Tactical  Exploitation 

Command 

TECOM 

Test  And  Evaluation  Command 

TACON 

Tactical  Control 

TEL 

T  ransporter/Erector/Launcher 

TACP 

Tactical  Air  Control  Party 

TELE-MED 

Television  Medicine 

TAC  SAT 

Tactical  Satellite 

TEM/OPS 

Terrain  Evaluation 

TACSOP 

Tactical  Standing  Operating 

Module/Obstacle  Planning 

Procedure 

System 

TADARS 

Target  Acquisition,  Designation 

TENCAP 

Tactical  Exploitation  Of  National 

And  Reconnaissance  System 

Capabilities 

TAEDP 

Total  Army  Equipment 

TEXCOM 

Test  And  Experimentation 

Distribution  Program 

Command 

TAFSM 

Target  Acquisition  Fire  Support 

TEXCOM 

U.S.  Army  Test  And 

Model 

TF 

Experimentation  Command 

Task  Force 
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TFFSE 

Task  Force  Fire  Support  Element 

TRAC-WSMR 

TRADOC  Requirements 

TFAWE 

Taskforce,  Advanced 

Analysis  Center  -  White  Sands 

Warfighting  Experiment 

Missile  Range  (White  Sands, 

TFC 

Traffic 

NM) 

THAAD 

Theater  High-Altitude  Area 

TRADOC 

U.S.  Army  Training  And 

Defense 

Doctrine  Command 

TI 

Tactical  Internet 

TRANS 

Transportation 

TIG 

The  Inspector  General 

TRANSCOM 

U.S.  Army  Transportation 

TIMS 

Tactical  Internet  Management 

Command 

System 

TRAS 

Training  Requirements  Analysis 

TIR 

Test  Incident  Report 

System 

TJAG 

The  Judge  Advocate  General 

TRC 

Type  Requisition  Code 

TLAM 

Tactical  Land  Attack  Missile 

TRICAP 

Triple  Capabilities 

TLAM 

Tomahawk  Land  Attack  Missile 

TRL 

Technology  Research  Levels  (1- 

TLP 

Troop  Leading  Procedure 

7) 

TM 

(1)  Theater  Missile;  (2) 

TRN 

Terrain 

Technical  Manual 

TRP 

Troop 

TM 

Team 

TRW 

Thompson  Ramo  Wooldridge 

TMAS 

(1)  Thermal  Mine  Avoidance 

Company 

System;  (2)  Tank  Main 

TSCD 

Targeting  Station  Control  And 

Armament  System 

Display 

TMD 

Tactical  Munitions  Dispenser 

TSFO 

Training  Set  Fire  Observation 

TMD 

Theater  Missile  Defense 

TSG 

The  Surgeon  General 

TMDE 

Test,  Measurement,  And 

TSM 

TRADOC  System  Manager 

Diagnostic  Equipment 

TSMO 

Threat  Support  Management 

TMM 

Target  Management  Matrix 

Office 

TMS 

Technology  Maturation  Strategy 

TSOP 

Tactical  Standing  Operating 

TNG 

Training 

Procedure 

TNGDEV 

Training  Developer;  Training 

TSS 

Topographic  Support  Systems 

Development 

TSS 

Target  Selection  Standards 

TOA 

Transfer  Of  Authority 

TSTAR 

Tomahak  Stops  The  Attacking 

TOA 

Total  Obligational  Authority 

Regiment 

TOC 

Tactical  Operations  Center 

TTHS 

Transients,  Trainees,  Holdees, 

TOE 

Table  Of  Organization  And 

And  Students 

Equipment 

TTL 

Time  To  Live 

TOM-D 

Training,  Operation, 

TTP 

Tactics,  Techniques,  And 

Mobilization,  And  Deployment 

Procedures 

TOPMIS 

Total  Officer  Personnel 

TU 

Traversing  Unit 

Management  Information  System 

TUAV 

Tactical  Unmanned  Aerial 

TOPO 

Topographical 

Vehicle 

TOPSS 

The  Officer  Projection  Specialty 

TUAV 

Tactical  Unattended  Vehicle 

System 

U.S. 

United  States 

TOR 

Terms  Of  Reference 

UAD 

Updated  Authorizations 

TOW 

Tube-Launched,  Optically 

Document 

Tracked,  Wire-Guided  Missile 

UAM 

Unattended  Munitions 

TPF 

Total  Package  Fielding 

UAV 

Unmanned  Aerial  Vehicle 

TPFDD 

Time-Phased  Force  Deployment 

UAV-CR 

Unmanned  Aerial  Vehicle 

Data 

UAV-E 

Unmanned  Aerial  Vehicle 

TPFDL 

Time-Phased  Force  Deployment 

UAV-SR 

Unmanned  Aerial  Vehicle-Short 

List 

Range 

TPG 

Troop  Program  Guidance 

UAV-SR 

Unmanned  Aerial  Vehicle 

TPM 

Total  Preventive  Maintenance 

UCP 

Unified  Command  Plan 

TPS 

Target  Processing  Section 

UDP 

User  Datagram  Protocol 

TPU 

Troop  Program  Unit 

UGS 

Unmanned  Ground  Systems 

TRAC 

TRADOC  Analysis  Command 

UGS  (BUGS) 

Bug-Sized  UGS 

(U.S.  Army) 

UGS  (MUGS) 

Mobile  UGS 
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UGS  (SLUGS) 

Stationary  UGS 

USD(A&T) 

Under  Secretary  Of  Defense  For 

UGV 

Unmanned  Ground  Vehicles 

Acquisition  And  Technology 

UH 

Utility  Flelicopter 

USD(Comptroller)  Under  Secretary  Of  Defense  For 

UIC 

Unit  Identification  Code 

Comptroller 

UK 

United  Kingdom 

USD(P&R) 

Under  Secretary  Of  Defense  For 

ULLS 

Unit  Level  Logistics  System 

Personnel  And  Readiness 

UML 

Unified  Modeling  Language 

USD(P) 

Under  Secretary  Of  Defense  For 

UMMCA 

Unspecified  Minor  Military 

Policy 

Construction,  Army 

USEUCOM 

U.S.  European  Command 

UMS 

Unmanned  Systems 

USMC 

U.S.  Marine  Corps 

UN 

United  Nations 

USMTF 

U.S.  Message  Text  Format 

UNAAF 

Unified  Action  Armed  Forces 

USMTF 

U.S.  Message  Text  Format 

UPH 

Unaccompanied  Personnel 

USN 

U.S.  Navy 

Housing 

USPACOM 

U.S.  Pacific  Command 

URN 

Unit  Reference  Number 

USPFO 

U.S.  Property  And  Fiscal  Officer 

URS 

Unit  Reference  Sheet 

USPHS 

U.S.  Public  Health  Service 

US 

United  States 

USR 

Unit  Status  Report 

USA 

U.S.  Army 

UTA 

Unit  Training  Assembly 

USA 

Under  Secretary  Of  The  Army 

UTM 

Universal  Transverse  Mercator 

USAARMC 

U.S.  Army  Armor  Center  And 

UTR 

Unit  Task  Reorganization 

School 

UW 

Unconventional  Warfare 

USAGE 

U.S.  Army  Corps  Of  Engineers 

VAAV 

Vehicle  Autonomous  Air  Vehicle 

USACGSC 

U.S.  Army  Command  And 

VCJCS 

Vice  Chairman,  Joint  Chiefs  Of 

General  Staff  College 

Staff 

USACMLS 

U.  S.  Army  Chemical  School 

VCSA 

Vice  Chief  Of  Staff,  Army 

USAEC 

U.S.  Army  Evaluation  Center 

VEH 

Vehicle 

USAF 

U.S.  Air  Force 

VET 

Veterinarian 

USAFAS 

U.S.  Army  Field  Artillery  School 

VGAS 

Vertical  Gun  For  Advanced 

USAFMSA 

U.S.  Army  Force  Management 

Ships  (Navy) 

Support  Agency 

VHSIC 

Very  High-Speed  Integrated 

USAIC&FH 

U.S.  Army  Intelligence  Center 

Circuit 

And  Fort  Huachuca 

VIC 

Vector  In  Command 

USAINSCOM 

U.S.  Army  Intelligence  And 

VLS 

Vertical  Launch  System 

Security  Command 

VMF 

Variable  Message  Format 

USAISC 

U.  S.  Army  Information  Systems 

VMS 

Virtual  Memory  System 

Command 

VSEL 

Vickers  Shipbuilding  & 

USAMEDCOM 

U.S.  Army  Medical  Command 

Engineering  Limited 

USAOTC 

U.S.  Army  Operational  Test 

VSTOL 

Vertical  Short  Takeoffllanding 

Command 

VT 

Variable  Time 

USAOTC 

U.S.  Army  Operational  Test 

VTOL 

Vertical  Take-Off  And  Landing 

Command 

VV&A 

Verification,  Validation,  And 

USAR 

U.S.  Army  Reserve 

Accreditation 

USARC 

U.S.  Army  Reserve  Command 

w/ 

With 

USAREC 

U.S.  Army  Recruiting  Command 

WAM 

Wide  Area  Mine 

USAREUR 

U.S.  Army  Europe 

WARF 

Wartime  Active  Replacement 

USARF 

U.S.  Army  Reserve  Force 

Factors 

USARPAC 

U.S.  Army  Pacific 

WARLINK 

Warrior  Link 

USASMDC 

U.S.  Army  Space  And  Missile 

WARNO 

Warning  Order 

Defense  Command 

WARSIM  2000 

Warfighter  Simulation  2000 

USASOC 

U.S.  Army  Special  Operations 

WECM 

Warfighter  Electronic  Collection 

Command 

And  Mapping  (Operational 

USATIC 

U.  S.  Army  Technology 

Concept) 

Integration  Center 

WFLA 

Warfighting  Lens  Analysis 

use 

U.S.  Code 

WHL 

Wheeled 

USCINCPAC 

U.S.  Commander-In-Chief, 

Pacific  Command 

WIA 

Wounded  In  Action 
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WIDD 

Warfighting  Integration  & 
Development  Directorate 

WIN 

WWMCCS  Intercomputer 
Network 

WINS 

Wireless  Intelligence  Network 

WINT-T 

Warfighter  Information  Network 
-  Tactical 

WMD 

Weapons  Of  Mass  Destruction 

woe 

Wing  Operation  Center 

WOF 

Worthy  Of  Fielding 

WP 

White  Phosphorous 

WPNS 

Weapons 

WRAP 

Warfighter  Rapid  Acquisition 
Program 

WRSI 

War  Reserve  Secondary  Items 

WSMR 

White  Sands  Missile  Range 
(White  Sands,  NM) 

WSRO 

Weapon  System  Replacement 
Operation 

WUIS 

Work  Unit  Information 
Summaries 

WWI 

World  War  I 

WWII 

World  War  II 

WWMCCS 

Worldwide  Military  Command 
And  Control  System 

XM30  LCPK 

Missile  Low-Cost  Precision  Kill 

XM984 

Extended-Range  Mortar 
Cartridge 

XO 

Executive  Officer 

Z 

Zulu 
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Appendix  C  -  U.S.  Anny  BOS  Diagrams. 

This  appendix  contains  a  graphical  representation  (Gibbings,  1991)  of  the  sub  function 
structure  of  each  of  the  seven  U.S.  Army  BOS’s.  The  BOS  do  not  represent  Army  branches  or 
proponents,  but  rather,  any  type  organization  that  relates  to  one  or  more  of  the  seven  BOS. 
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Figure  45  -  Maneuver  BOS 
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Figure  46  -  Fire  Support  BOS 
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Figure  47  -  Air  Defense  BOS 
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Figure  48  -  Command  And  Control  BOS 
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Figure  49  -  Intelligence  BOS 
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Figure  50  -  Mobility  And  Survivability  BOS  -  Part  1 
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Figure  51  -  Mobility  And  Survivability  BOS  -  Part  2 
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Figure  52  -  Combat  Service  Support  BOS  -  Part  1 
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Figure  54  -  Combat  Service  Support  BOS  -  Part  3 


Appendix  D  -  Critical  Incident  Interview  Guide. 

This  interview  guide  provides  a  suggested  framework  for  conducting  interviews  of 
system  operators  following  the  performance  of  a  critical  incident  (Randel  et  ah,  1996).  It  is 
useful  in  developing  data  as  part  of  a  cognitive  task  analysis  while  investigating  intensive 
cognitive  tasks  such  as  decision  making. 


Critical  Incident  Interview  Guide 

(Randel  et  ah,  1996) 

A.  Describe  the  purpose  of  the  interview:  “We  want  to  learn  how  you  make  decisions  while 
working  on  your  job.  Please  respond  accurately  and  completely  and  take  your  time.  I  would  like 
to  use  the  tape  recorder  in  case  I  can’t  write  everything  down,  if  that  is  OK  with  you.  No  one 
outside  this  research  group  will  hear  the  tape.” 

B.  Give  the  interviewees  some  paper  and  a  pencil  in  case  they  want  to  make  a  diagram. 

C.  Start  the  interview. 

1 .  Select  incident. 

“I  would  like  you  to  discuss  some  difficult  decisions  or  events  that  occurred  while 
you  were  performing  your  job.  This  could  be  a  situation  where  an  error  was  likely,  or 
did  in  fact  occur.  Let’s  start  with  one  such  event.” 

2.  Obtain  unstructured  incident  account  in  the  form  of  a  time  line. 

‘describe  the  incident  from  start  to  completion.  I  would  like  to  reconstruct  the 
events  in  the  fonn  of  a  time  line  that  establishes  the  sequence  of  each  event.” 

Repeat  the  incident  back  to  the  interviewee  to  make  sure  you  have  it  correct. 

3.  Decision  point  identification. 

“Now  I  would  like  you  to  help  me  identify  specific  decision  points  on  the  time 
line.  A  decision  point  occurs  when  someone  else  might  have  s\chosen  to  act 
differently.” 

4.  Decision  point  probing. 

For  each  decision  point,  use  the  following  probes. 

a.  Errors.  “If  an  error  occurred,  what  was  the  error?” 

Hypothetical.  “If  an  error  occurred,  how  should  the  incident  have  proceeded?” 

Missing  data.  “What  would  have  helped  you  make  the  decision?  What  data  or 
information  was  missing?” 
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Error  avoidance.  “How  could  the  error  have  been  avoided?” 

b.  Cues/Situation  Awareness.  “What  were  you  seeing  or  hearing  that  caused  you  to 
make  this  particular  decision?” 

c.  Comparison  to  novice  (if  interviewing  an  expert).  “How  might  someone  with  less 
experience  have  responded?” 

d.  Knowledge.  “What  information  did  you  use  in  making  this  decision,  and  how  was 
it  obtained?” 

e.  Decision  options. 

(1)  “What  other  choices  did  you  consider  or  were  available  to  you?” 

{If  none,  do  not  proceed  with  options.} 

List  other  choices: 

(a)  Restate  first  rejected  option. 

“What  was  the  reason  for  rejecting  it?” 

(b)  Restate  each  rejected  option  and  ask  the  reason  for  rejecting  it. 

(2)  Classification  of  the  types  of  options  elicited.  “Were  the  options  you 
considered  standard  or  constructed?” 

(a)  Standard  -  ones  you  are  commonly  taught  to  consider. 

(b)  Constructed  -  created  by  you  or  changed  from  the  standard  in  some  way. 
(If  so,  “What  experience  was  necessary  to  generate  this  option  {personal 
operator  experience,  special  training,  etc.}?”) 

f.  Rule  of  thumb.  “If  a  new  operator  was  standing  beside  you  during  this  incident 
and  asked  you  what  rule  (advice)  you  were  following  when  you  made  this 
decision,  what  would  you  say?” 

5.  Additional  incidents.  “Can  you  recall  another  incident  that  was  difficult  or  where 
an  error  might  likely  have  occurred?”  If  so,  repeat  steps  2-5. 
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Appendix  E  -  Example  of  A  Brigade  Operations  Order. 

This  is  an  example  of  a  brigade  operations  order  that  was  used  in  one  of  the  exercises 
observed  by  this  dissertation  project.  Except  for  the  sanitization  of  names  and  locations 
reflecting  fictional  units,  countries,  and  locations  that  was  done  by  the  unit  to  allow  the  exercise 
to  be  conducted  in  an  unclassified  arena,  this  is  representative  of  the  type  of  written  commands 
that  are  generated  prior  to  actual  combat  missions  where  the  commander’s  guidance  and  intent  is 
transmitted  using  the  basic  5  paragraph  operations  order  fonnat. 


Copy _ of _ copies 

1BCT  Headquarters,  9 AD 
FT  HOOD,  TX 
21  1000  MAR  02 


1st  BCT  OPLAN  02-03-01 

Task  Organization:  See  Annex  A:  Task  Organization. 

1.  SITUATION. 

a.  Enemy  Forces. 

(1)  Current  Situation:  Intelligence  reports  indicate  that  Gordian  SPF  UW 
elements  from  Gordo  have  infiltrated  Kazar.  The  approximately  800  infiltrated  personnel  have 
attacked  government  buildings,  conducted  arson,  kidnappings,  and  ethnic  cleansing  in  the  region. 
Several  large3  caches  of  small  arms,  communication  equipment,  and  mines  have  been  discovered 
The  Gordian  conventional  military  forces  are  capable  of  attacking  with  five  brigade-size 
elements  from  1st  Corps  within  three  to  six  days.  The  Gordian  conventional  forces  have  made  no 
move  from  their  current  positions,  but  intelligence  sources  indicate  that  there  has  been  increased 
communication  activity  between  Unconventional  Forces  within  Kazar  to  elements  in  Gordo 
military  units. 

(2)  Enemy  most  likely  Course  of  action.  TBP 

b.  Friendly  Forces. 

(1)  JTF. 

(a)  Mission.  Joint  Task  Force  KAZAR  deploys  to  Kazar  and  conducts 
military  operations  to  restore  stability  to  Kazar.  On  order,  conduct  hand  over  to  United  Nations 
Peacekeeping  forces. 

(b)  Intent.  Neutralize  Gordian  UW  activities  and  ethnic  Skandian 
hostilities  directed  against  ethnic  Gordians  in  Kazar,  and  be  prepared  to  defeat  cross-border 
incursions  by  Gordian  Forces. 
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(c)  Concept.  Omitted 

(2)  ARFOR. 

(a)  Mission.  On  order,  the  ARFOR  deploys  to  conduct  operations  in 
KAZAR,  conducts  limited  reception,  staging,  onward  movement  and  integration  (RSOI)  defeats 
Gordian  unconventional  forces  and  deters  employment  of  Gordian  conventional  forces,  in  order 
to  stabilize  KAZAR.  On  order,  conducts  hand-over  to  U.N.  Peacekeeping  forces  and  deploys. 

(b)  Intent.  I  intend  to  deploy  ARFOR,  in  conjunction  with  the  JTF,  as 
quickly  as  possible  to  stabilize  the  situation  in  Kazar.  I  will  use  the  l/325th  Parachute  Infantry 
Regiment  to  secure  Pristina  (Georgetown)  Airfield.  1st  BCT  will  secure  the  city  of  Pristina  and 
immediate  environs,  eliminating  all  aspects  of  the  paramilitary  threat.  Follow-on  forces  will 
operate  in  the  West  and  North,  eliminating  insurgent  forces  and  deterring  possible  Gordian 
incursions  in  Kazar.  Once  I  am  satisfied  that  the  situation  is  stabilized,  I  will  recommend  to  the 
JTF  the  ARFOR  transfer  control  of  Kazar  to  U.N.  forces  and  redeploy  to  home  station. 

(c)  Concept  of  operations.  This  is  a  four  phase  operation: 

-  Phase  I  Entry  Operations 

-  Phase  II  Defeat  Gordian  SPF  and  deter  Gordian  Conventional 

Attack 

-  Phase  III  Transfer  Control  to  Coalition  Peacekeeping  Forces 

-  Phase  IV  Redeployment 

(d)  Assumptions. 

1 .  SPF  activity  remains  low  in  AOR  #5  (Brigade  Rear  Area. 

2.  IBCT  and  1st  KAZAR  are  able  to  protect  West  and  East  flanks. 

(3)  Adjacent  Units. 

(a)  Left/West.  IBCT:  Remaining  units  of  IBCT  move  north  from  Skopje 
(Austin)  assembly  areas  to  Kosovska/Mitrovica,  establish  presence,  defeat  SPF  in  sector,  and 
prepare  to  counter  incursions  by  1st  Corps  along  the  international  border  on  a  line  generally  from 
VUCIFRN  PV0546,  Kosovska/Mitrovica  NV7838,  and  Colorado  River  NV5329. 

(b)  Right/East.  1st  Kazarian  Brigade 

(c)  Front/North.  Phase  II:  1-10  CAV  Relocate  assets  to  Pec  Airfield  in 
order  to  support  IBCT  along  the  international  border  (IB).  Conduct  zone  recon  forward  of  Phase 
Line  Kansas  to  GORDO/KAZAR  IB  to  identify  threat  forces  in  zone,  and  to  provide  early 
warning  to  deploying  brigades.  Upon  consolidation  of  objective  (OBJ)  JACK  by  IBCT,  provide 
recon  of  IB  between  VUCIFRN  on  the  east  and  the  Colorado  River  on  the  west.  Prepare  to 
engage  advancing  GORDO  units  in  IBCT  sector.  Priority  of  support  is  to  IBCT. 


310 


(d)  Rear/South.  Phase  II:  2  BCT  Upon  arrival  at  Skopje,  prepare  to  move 
north  to  Pristina  and  assume  control  from  l/325th  PIR.  Prepare  to  assume  overwatch  over  area 
of  approach  (AA)1  and  A  A  2.  Prepare  to  conduct  operations  to  defeat  Gordian  SPF  units  in 
sector. 


2.  MISSION.  On  order,  1st  BCT  occupies  AO  Raider  and  defeats  GORDIAN  Special 
Forces  units  in  sector  in  order  to  deter  GORDIAN  conventional  forces.  Be  prepared  to 
conduct  defensive  operations  to  defeat  Gordian  conventional  forces  within  AO  Raider.  On 
order,  conducts  hand-over  to  U.N.  Peacekeeping  forces  and  re-deploy. 

3.  EXECUTION. 

Commander’s  Intent. 

PURPOSE:  The  purpose  of  this  operation  is  to  stabilize  the  situation  in  AO  Raider. 

KEY  TASKS:  Occupy  AO  Raider 

Defeat  Gordian  Special  Police  Force  (SPF)  units 

Deter  Gordian  conventional  attack 

Be  prepared  to  defeat  conventional  forces  in  sector. 

ENDSTATE:  Gordian  SPF  units  rendered  in-effective,  situation  stabilized  in  AO  Raider. 
1st  BCT  prepared  to  defend  against  a  Gordian  military  attack,  United  Nations  Peacekeeping  units 
in  sector  preparing  for  battle  handover. 

a.  Concept  of  the  Operation.  The  decisive  point  of  this  operation  is  completing  the 

Brigade’s  show  of  force  by  occupying  AO  Raider.  On  order,  1st  BCT,  4^  ID  (M)  secures 
KAZAR  City  and  establishes  AORs  (Areas  of  Responsibility)  in  sector  in  order  to  defeat 
Gordian  SPF  units  and  deter  elements  of  Gordian  conventional  forces.  The  brigade  occupies 
KAZAR  city  by  establishing  task  force  AORs  with  two  Task  Forces  forward  and  one  back. 
CLOSE:  G/10  CAV  occupies  the  1  KM  buffer  zone  to  identify  Gordian  SPF  and  insurgent 
forces  and  provide  early  warning  to  the  BCT.  4th  MP  (-)  (1st  and  2nd  Platoon  supports  civil 
authorities  in  the  Brigade’s  exclusion  zone  in  conjunction  with  the  local  police  in  order  to 
prevent  civil  disturbances  in  sector.  TF  1-22,  the  main  effort,  occupies  AOR  Regular  in  sector  in 
order  deter  elements  of  Gordo  from  conducting  insurgent  operations.  TF  1-66,  supporting  effort 
1,  occupies  AOR  IRON  KNIGHT  in  sector  in  order  to  protect  the  flank  of  the  Brigade  main 
effort  and  protect  the  Brigade  Support  Area.  3-66  AR,  supporting  effort  2,  occupies  AOR 
LANCER  and  protects  the  eastern  flank  of  the  ME.  Fires  suppress  SPF’s  to  allow  freedom  of 
movement  to  the  BCT’s  within  AO  Raider.  REAR:  A/95th  MP  protects  the  Brigade  logistics 
and  lines  of  communication  in  order  to  prevent  insurgent  activities  in  the  Brigade  Rear  Areas. 

(1)  Scheme  of  Maneuver. 
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(a)  Phase  I  (Entry  Operations).  This  phase  begins  when  the  Brigade 
moves  north  to  Pristina  to  secure  the  city  and  immediate  environs,  eliminate  all  aspects  of 
paramilitary  threat  and  prepare  for  follow-on  missions.  This  phase  ends  with  the  completed 
battle  hand-over  with  the  lst/325  PIR.  (COMPLETED  as  of  24  MAR  02) 

(b)  Phase  II  (Defeat  Gordian  SPF  and  deter  a  GORDIAN  conventional 
attack).  This  phase  begins  with  the  Brigade’s  occupation  of  AO  Raider.  Initially,  the  BRT 
occupies  OPs  along  the  1  KM  buffer  zone  along  the  international  boundary  between  KAZAR 
and  GORDO.  The  Brigade  occupies  AO  Raider  in  order  to  defeat  GORDIAN  SPF  and  deter  a 
GORDIAN  conventional  attack.  DEEP:  The  brigade  employs  information  operations  to  isolate, 
disrupt,  and  demoralize  insurgent  forces  in  zone  and  influence  GORDIAN  military  leadership  to 
believe  that  ARFOR  is  capable  of  defeating  attacks  into  KAZAR.  Each  TF  has  a  CA/PSYOP/CI 
team  in  order  to  conduct  information  operations  in  their  TF  AOR.  TFs  will  gain  the  confidence 
of  local  officials  to  include  village  mayors  and  leaders,  police  chiefs,  religious  leaders,  business 
leaders,  media  and  local  citizens.  TFs  will  identify  NGOs  (Non  Governmental  Organizations) 
and  PVOs  (Private  Volunteer  Organizations)  in  their  AORs  and  provide  reasonable  support  as 
appropriate.  CLOSE:  AO  Raider  consist  of  Task  Forces  occupying  AORs.  The  TFs  will  defeat 
GORDIAN  SPF  teams  by  patrolling  designated  routes,  establishing  check  points  as  necessary, 
seizing  contraband,  clearing  routes  of  mines,  and  preventing  crimes  against  humanity.  Each  TF 
will  maintain  one  company  positioned  on  camps  near  the  international  border  as  a  show  of  force 
in  order  to  deter  GORDIAN  attack  and  at  least  one  tank  or  mech  platoon  as  a  QRF.  TFs  will 
patrol  designated  routes  twice  each  day.  Mission  Essential  Vulnerable  Areas  (MEVAs)  will  be 
checked  at  least  twice  a  day.  At  all  times,  TFs  will  detain  black  (wanted  for  war  crimes  and 
terrorist  activity)  and  grey  (criminal  activity)  list  personnel  in  order  to  stabilize  the  region.  TF  1- 
22,  the  main  effort,  occupies  AOR  REGULAR  north  of  KAZAR  CITY  in  order  to  defeat 
GORDAIN  SPF  and  deter  GORDIAN  conventional  attack.  TF  1-66,  supporting  effort,  occupies 
AOR  IRON  KNIGHT  in  order  to  protect  the  flank  of  the  Brigade  main  effort  and  protect  the 
Brigade  Support  Area.  3-66  AR,  supporting  effort,  occupies  AOR  LANCER  to  protect  the 
eastern  flank  of  the  ME.  The  BRT  occupies  designated  OPS  along  the  1KM  zone  to  observe 
NAIs  IOT  provide  early  warning  of  cross-border  invasion  routes.  The  brigade  accepts  risks  in  the 
south  with  only  A/95  MP  CO  to  patrol  MSRs  in  AOR  5  (Brigade  Rear  Area).  TF  1-66  is 
prepared  to  provide  a  QRF  to  reinforce  A/95  MPs  as  needed  in  AOR  5.  The  4th  MP  (-)  conducts 
multinational  patrols  with  local  police  forces  in  OBJ  KING  (KAZAR  CITY).  TFs  will  have  a 
howitzer  battery  in  their  AOR  and  maintain  a  minimum  of  platoon  fire  base  (3  guns)  and  one  hot 
gun  (ready  to  fire  immediately)  at  all  times.  TFs  have  an  engineer  company  in  their  AOR  to 
conduct  mobility  operations  to  clear  routes  and  areas  of  mines  and  sustain  LOCs  and 
survivability  operations  to  improve  force  protection  measures  in  camps.  The  brigade  is  to  be 
prepared  to  execute  defensive  CONPLANS  in  order  to  defeat  GORDIAN  conventional 
aggression.  REAR:  Each  TF  will  maintain  their  FSE  in  their  TF  base  camp.  This  phase  ends 
with  the  transfer  of  control  to  the  U.N.  Peacekeeping  forces. 

(c)  Phase  III  (Transfer  Control  to  Coalition  Peacekeeping  Force).  This 
phase  begins  with  the  transfer  of  control  of  KAZAR  city  to  the  U.N.  peacekeeping  forces.  This 
phase  ends  with  the  completion  of  transfer  of  control  to  U.N.  peacekeeping  forces. 
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(d)  Phase  IV  (Redeployment).  This  phase  begins  with  the  redeployment 
of  1st  BCT  soldiers  and  equipment.  This  phase  ends  with  the  safe  return  of  all  1st  BCT  soldiers 
and  equipment  to  the  continental  U.S. 

(2)  Fires.  See  Annex  D:  Fire  Support. 

(3)  Engineer.  Engineers  support  all  phases  with  mobility,  survivability,  and 
sustainment  engineering  operations.  Engineer  companies  task  organized  with  maneuver 
battalions  ensure  freedom  of  maneuver  by  clearing  mines  and  obstacles  off  routes  and 
conducting  limited  repairs  to  road  surfaces  and  fording  sites  as  required.  High  Value  Assets 
(HVAs)  are  supported  with  survivability  assets.  Host  nation  contracted  horizontal  construction 
assets  under  the  control  of  the  299  EN  BN  HQ  are  prepared  to  conduct  airfield  and  MSR  repair 
operations.  Engineers  will  employ  the  759th  ORD  (EOD)  to  identify,  mark,  report,  and  O/O 
destroy  unexploded  ordnance  (UXOs)  and  coordinate  for  EOD  support.  All  identified  obstacles 
and  UXOs  not  cleared  will  be  turned  over  to  A/307  EN  (ABN).  Engineers  are  prepared  to 
transition  to  countermobility  operations  in  the  event  Gordian  conventional  forces  are  identified 
crossing  the  border  into  KAZAR. 

Essential  Mobility  Survivability  Tasks: 

T:  Provide  mobility  to  1BCT  forces. 

P:  Facilitate  northward  movement  into  AO  Raider. 

M:  Clearing  and  maintaining  open  routes  in  AO  Raider. 

E:  SPF  don”t  impede  1BCT  occupation  of  AO  Raider  and  support  positive  relations  with 
local  Kazarians. 

(4)  Reconnaissance  &  Surveillance.  See  Annex  L:  R&S. 

(5)  Air  Defense.  A/1-44  ADA  (+)  integrates  and  deploys  with  1st  BCT,  4ID  (M) 
to  the  Province  of  KAZAR  and  provides  air  defense,  surveillance  and  general  support  throughout 
the  established  task  force  AORs.  One  Linebacker  Platoon  (l/A/1-44  ADA)  with  four  Linebacker 
fire  units  provides  direct  support  to  TF  1-22  in  AOR  Regular.  Fire  units  deploy  with  supported 
CO/TMs  and  monitor  AAAs  within  AOR  Regular.  Air  defense  coverage  in  AOR  Regular  is 
weighted  forward  and  to  the  east  monitoring  the  International  Border  and  buffer  zone  in  order  to 
detect  possible  RW  insurgency  into  the  Province  of  KAZAR.  One  Linebacker  Platoon  (2/A/1-44 
ADA)  with  four  Linebacker  fire  units  provides  direct  support  to  3-66  in  AOR  Lancer.  Air 
defense  coverage  in  AOR  Lancer  is  weighted  to  the  northeast  along  AAAs  2a  (along  E  Range 
Road  heading  south  into  KAZAR)  and  2b  (moving  through  the  Belton  Reservoir  from  the 
northeast  to  the  southwest)  and  in  the  east  along  2c  (following  US  Hwy  190  from  the  1-35 
corridor  to  vie  KAZAR  CITY).  One  Avenger  section  ( 1/3/A/1-44  ADA)  provides  GS  AD 
coverage  vie  BSA  and  Robert  Gray  Army  Airfield  (RGAAF)  to  TF  1-66  located  in  AOR  IRON 
KNIGHT.  One  Avenger  section  (2/3/A/ 1-44  ADA)  remains  under  BCT  control  and  provides  GS 
AD  for  HVTs  (Killeen  Municipal  Airport  and  US  HWY  190,  Hood  Army  Airfield,  BCT  TOC) 
vie  KAZAR  CITY  and  the  BCT  Exclusion  Zone.  Sentinel  1  sets  vicinity  West  Fort  Hood  and 
provides  EW  coverage  vie  AORs  REGULAR  and  IRON  KNIGHT  and  associated  AAAs. 
Sentinel  2  sets  vicinity  Hood  Army  Airfield  and  Airfield  Lake  and  provides  EW  coverage  vie 
AOR  LANCER,  the  BCT  exclusion  zone  and  associated  AAAs. 
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UNIT 

SUPPORTED  UNIT 

LOCATION 

1 /A/ 1-44  ADA 

TF  1-22 

1  LB  DS  B/3-66  AOR  Regular 

2  LB  DS  A/1-22  AOR  Regular 

1  LB  DS  C/1-66  AOR  Regular 

2/A/ 1-44  ADA 

3-66 

1/2/A/1-44  (2  LB)  DS  B/3-66 
AOR  Lancer 

2/2/A/1-44  (2  LB)  DS  A/3-66 
AOR  Lancer 

1/3/A/l -44  ADA 

TF  1-66 

1  AVENGER  VIC  PV  092388 

1  AVENGER  VIC  PV  1 16405 
(Crossville  Mtn) 

1  AVENGER  VIC  PV  088362 
(Radar  Hill) 

2/3/A/ 1-44  ADA 

BCT,  KAZAR 

CITY 

1  AVENGER  VIC  PV  179488 
(Blackwell  Gap) 

1  AVENGER  VIC  PV  262463 
(Hood  AAF) 

1  AVENGER  VIC  PV  315379 
(Hill  258) 

SENTINEL  1 

BCT 

PV  156412 

SENTINEL  2 

BCT 

PV  241464 

b.  Tasks  to  Subordinate  Units. 


(1)  TF  1-22  (ME) 

(a)  Phase  I. 

1 .  Occupy  Pristina  in  order  to  support  stability  operations. 

2.  O/O  conduct  battle-hand  over  with  1  st/325t  PIR  of  Pristina 

Airfield. 


activities. 


necessary. 


bases. 


(b)  Phase  II 

1 .  Occupy  AOR  REGULAR  in  order  to  deter  Gordo  insurgent 

2.  Establish  Task  Force  Base  Camp  within  AOR  REGULAR. 

3.  Establish  CPs  and  Patrol  Bases  within  AOR  REGULAR  as 

4.  Conduct  operations  to  defeat  Gordian  SPF  units  in  sector. 

5.  Conduct  platoon  size  patrols  within  AOR  REGULAR. 

6.  Establish  checkpoints  in  AOR  REGULAR  as  required. 

7.  Limit  civilian  access  to  Task  Force  Base  Camps,  CPs,  and  patrol 


314 


8.  Stop  and  check  civilians  at  all  checkpoints  IAW  Annex  E. 

9.  Seize  contraband  IAW  Annex  E. 

10.  Prevent  crimes  against  humanity  IAW  Annex  E. 

1 1 .  Employ  information  operations  to  isolate,  disrupt,  and 
demoralize  insurgent  forces  in  zone. 

12.  Gain  confidence  of  the  local  officials  to  include  village  mayors 
and  leaders,  police  chiefs,  religious  leaders,  business  leaders,  media  and  local  citizens. 

13.  Identify  NGOs  and  PVOs  in  AORs  and  provide  reasonable 

support  as  appropriate. 

14.  Clear  ROUTES  BLUE  and  WHITE  in  AORs  IAW  Annex  L 


(R&S) 

region. 

REGULAR 


15.  Detain  all  black  and  grey  list  personnel  in  order  stabilize  the 

16.  Provide  Platoon  QRF  to  destroy  SPF  forces  in  AOR 


17.  BPT  defeat  conventional  Gordian  Forces  in  sector  in  EA 
METZ,  BERLIN,  and  DRAGOON  IOT  deter  a  Gordian  Conventional  Attack. 


(c)  Phase  III.  O/O  conduct  hand  over  of  KAZAR  city  with  U.N. 

Peacekeeping  Forces 

(d)  Phase  IV.  0/0  conduct  redeployment. 

(2)  TF  1-66  (SE  1) 


(a)  Phase  I.  Occupy  Pristina  in  order  to  support  stability  operations. 


activities. 


(b)  Phase  II 

1 .  Occupy  AOR  IRON  KNIGHT  in  order  to  deter  Gordo  insurgent 

2.  Establish  Task  Force  Base  Camp  within  AOR  IRON  KNIGHT. 

3.  Establish  CPs  and  Patrol  Bases  within  AOR  IRON  KNIGHT  as 


necessary. 

4.  Conduct  operations  to  defeat  Gordian  SPF  units  in  sector. 

5.  Conduct  platoon  size  patrols  within  AOR  IRON  KNIGHT. 

6.  Establish  checkpoints  in  AOR  IRON  KNIGHT  as  required. 

7.  Limit  civilian  access  to  Task  Force  Base  Camps,  CPs,  and  patrol 

bases. 

8.  Stop  and  check  civilians  at  all  checkpoints  IAW  Annex  E. 

9.  Seize  contraband  IAW  Annex  E. 

10.  Prevent  crimes  against  humanity  IAW  Annex  E. 

1 1 .  Employ  information  operations  to  isolate,  disrupt,  and 
demoralize  insurgent  forces  in  zone. 

12.  Gain  confidence  of  the  local  officials  to  include  village  mayors 
and  leaders,  police  chiefs,  religious  leaders,  business  leaders,  media  and  local  citizens. 

13.  Identify  NGOs  and  PVOs  in  AORs  and  provide  reasonable 

support  as  appropriate. 
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(R&S) 

region. 


14.  Patrol  ROUTES  Yellow  in  AORs  LANCER  IAW  Annex  L 

15.  Detain  all  black  and  grey  list  personnel  in  order  stabilize  the 

16.  BPT  defeat  conventional  Gordian  Forces  in  EA  SICILY  and 


EZ  ANZIO  IOT  deter  a  Gordian  Conventional  Attack. 


(c)  Phase  III.  O/O  conduct  hand  over  of  KAZAR  city  with  U.N. 

Peacekeeping  Forces 

(d)  Phase  IV.  0/0  conduct  redeployment. 


(3)  3-66  (SE  2) 

(a)  Phase  I  Occupy  Pristina  in  order  to  support  stability  operations. 

(b)  Phase  II 

1.  Occupy  AOR  LANCER  in  order  to  deter  Gordo  insurgent 

activities. 


2.  Establish  Task  Force  Base  Camp  within  AOR  LANCER. 

3.  Establish  CPs  and  Patrol  Bases  within  AOR  LANCER  as 


necessary. 

4.  Conduct  operations  to  defeat  Gordian  SPF  units  in  sector. 

5.  Conduct  platoon  size  patrols  within  AOR  LANCER. 

6.  Establish  checkpoints  in  AOR  LANCER  as  required. 

7.  Limit  civilian  access  to  Task  Force  Base  Camps,  CPs,  and  patrol 

bases. 

8.  Stop  and  check  civilians  at  all  checkpoints  IAW  Annex  E. 

9.  Seize  contraband  IAW  Annex  E. 

10.  Prevent  crimes  against  humanity  IAW  Annex  E. 

1 1 .  Employ  information  operations  to  isolate,  disrupt,  and 
demoralize  insurgent  forces  in  zone. 

12.  Gain  confidence  of  the  local  officials  to  include  village  mayors 
and  leaders,  police  chiefs,  religious  leaders,  business  leaders,  media  and  local  citizens. 

13.  Identify  NGOs  and  PVOs  in  AORs  and  provide  reasonable 

support  as  appropriate. 

14.  Patrol  ROUTES  ORANGE  and  GREEN  in  AOR  LANCER 

IAW  Annex  L  (R&S) 

15.  Detain  all  black  and  grey  list  personnel  in  order  stabilize  the 


region. 


16.  BPT  defeat  conventional  Gordian  Forces  in  EA  OMAHA  IOT 


defeat  a  Gordian  conventional  attack  into  AO  RAIDER. 
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(c)  Phase  III.  0/0  conduct  hand  over  of  KAZAR  city  with  U.N. 

Peacekeeping  Forces 

(d)  Phase  IV.  0/0  conduct  redeployment. 


(4)  G/10  CAV 

(a)  Phase  I.  Conduct  zone  reconnaissance  north  of  PL  KANSAS. 

(b)  Phase  II. 

1 .  Occupy  OP  and  observe  NAIs  from  along  the  1  KM  buffer  zone 
IOT  provide  early  warning  to  the  BCT. 

2.  Conduct  platoon  size  patrols  along  ROUTE  RED. 

3.  Gain  confidence  of  the  local  officials  to  include  village  mayors 
and  leaders,  police  chiefs,  religious  leaders,  business  leaders,  media  and  local  citizens. 

4.  Identify  NGOs  and  PVOs  in  AORs  and  provide  reasonable 

support  as  appropriate. 

5.  Occupy  OPs  IAW  Annex  L  (R&S) 

6.  Observe  NAIs  IAW  Annex  L  (R&S) 

7.  BPT  screen  the  KAZAR/GORDO  border  in  the  event  of 

GORDO  conventional  attack. 

(c)  Phase  III.  0/0  conduct  hand  over  of  KAZAR  city  with  U.N. 

Peacekeeping  Forces 

(d)  Phase  IV.  0/0  conduct  redeployment, 
c.  Task  to  Combat  Support. 


(1)4-42  FA. 

(a)  Provide  DS  fires  to  1st  BCT  in  order  to  allow  freedom  of  movement 


with  AO  Raider. 


(b)  Provide  1  Paladin  Battery  to  each  Task  Force  AOR. 


(2)  299  EN. 


(a)  Provide  mobility  support  to  maneuver  battalions  to  include  limited 
MSR  and  ford  site  repairs. 

(b)  Provide  survivability  support  to  HVAs  (Q-36,  Q-37). 

(c)  0/0  direct  Host  Nation  horizontal  construction  assets  to  conduct 
airfield  and  MSR  repair. 

(d)  Identify,  mark,  record,  report,  and  0/0  destroy  all  UXOs. 

(e)  BPT  conduct  obstacle  turn-over  to  A/307  EN  BN  on  all  obstacles  and 

UXOs  not  cleared. 

(f)  Coordinate  for  an  EOD  team  and  provide  support  to  their  operations. 
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(g)  Track  status  of  all  MSRs,  ASRs,  and  stream  crossings  and  coordinate 

all  necessary  repairs. 

(3)  A/95th  MP.  Protect  the  Brigade  logistics  and  lines  of  communication  in  order 
to  prevent  insurgent  activities. 

(4)  A/1 04th  MI.  Provide  FM  RETRANS  between  GCS’s  at  the  BCT  TOC  and  the 
TUAV  launch/recovery  site. 

(5)  l/4th  MP.  Conduct  supports  of  civil  authorities  in  the  Brigade’s  exclusion 
zone  in  conjunction  with  the  local  police  in  order  to  prevent  civil  disturbances. 

c.  Coordinating  Instructions. 

(1)  Task  organization  effective  25  0900  MAR  01 

(2)  This  OPLAN  is  effective  upon  receipt. 

(3)  CCIR. 

(a)  PIR. 


PIR 

STATUS 

START  / 
STOP 

INDICATORS  /  REMARKS 

1 .  Where  will  SPF  attempt  to  disrupt  1  BCT 
freedom  of  movement? 

riots,  illegal  checkpoints,  sabotage,  SPF  HQ 
vie  KAZAR 

2.  Will  the  43d  attack  across  the 

International  Border  vie  NAI  R201,  R203, 
R132,  R133,  R1 33a,  R136,  R138,  R217, 
R217a? 

8  or  more  CBT  armored  vehicles  moving 
along  route  Black,  greater  than  a  Battery,  5  or 
MICLICS/ACES,  4  or  plows  moving  East 

3.  Is  the  54th  repositioning  vie  KURSUMLIJA 
NAI  R301? 

30(+)  APC,  M60RCGL,  M80ATGL 

4.  Is  the  25th  repositiong  South  East  of 
KURSUMLIJA  vie  NAI  R316,  R311,  R329? 

30(+)  APC,  M60RCGL,  M80ATGL 

operations  in  sector? 
operations? 


(b)  EEFI. 

1.  Location  of  TUAV  Launch  and  Recovery  sites. 

2.  Location  of  Brigade  TOC 

(c)  FFIR. 

1.  Pennission  from  ARFOR  commander  to  transition  to  defensive 

2.  Are  2  of  the  3  TF  able  to  transition  from  SASO  to  defensive 

(d)  CSIR. 
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1 .  Any  loss  of  a  sensitive  item. 

2.  SBF  or  terrorist  incident  in  AO  Raider. 

3.  Serious  incidents  or  death  of  a  BDE  soldier 

4.  TF  OR  rate  below  90%. 

5.  Demonstration  of  500  people  or  more  in  AO  Raider. 

6.  Crash  of  ARFOR  aircraft  in  AO  Raider. 

7.  ROUTES  not  patrolled  twice  daily. 

8.  Closure  of  an  MSR  for  more  than  2  hours. 

9.  Any  attack  on  a  MEVA. 

10.  Any  route  not  patrolled  within  required  standard. 

1 1 .  Arrival  of  any  ARFOR  or  higher  VIP. 

12.  Loss  of  contact  of  a  patrol  for  more  than  1  hour. 

(5)  Risk  reduction  and  control  Measures.  ARFOR  ground  forces  (less 
reconnaissance)  will  conduct  operations  in  no  smaller  than  platoon  size  elements.  Platoon=  3 
vehicles  with  mounted  automatic  weapons.  See  Annex  W. 

(6)  ROE.  See  Annex  E. 

(7)  Force  Protection.  The  ARFOR  HQ  will  coordinate  for  host  nation  support 
(HNS)  to  defend  APODs  and  SPODs.  Unit  commanders  should  conduct  risk  assessment  that 
consider  operational  as  well  as  environmental  factors. 

(8)  System  administrators  and  commanders  at  all  levels  will  ensure  compliance 
with  policies  and  procedures  to  maintain  Infonnation  Assurance. 

(9)  Commanders  and  staffs  at  all  levels  will  ensure  compliance  with  ARFOR 
Information  Management  Plan  to  support  development  of  the  common  operating  picture  and 
facilitation  of  operational  understanding  of  the  battlespace. 

(10)  Commander,  Joint  Task  Force  (CJTF)  retains  authority  for  emplacement  of 
all  obstacles,  conventional  or  FASCAM. 

(11)  All  requests  for  obstacle  emplacement  to  the  CJTF  must  include  specific 
details  on  the  NAI,  purpose,  trigger,  exact  location,  composition,  decision  points,  and  duration. 

(12)  Signal  assets  attached  to  maneuver  units  are  OPCON’d  for  movement, 
security,  and  sustainment.  Signal  units  remain  under  technical  control  (TECHCON)  of  the 
ARFOR  signal  battalion. 

(13)  Provide  additions  to  graphic  control  measures  or  any  changes  in  OPS  to 
Raider  Mike  on  Brigade  Command. 

(14)  Units  will  have  to  make  extensive  use  of  their  organic  RETRANS  teams  due 
to  the  severe  terrain.  Ensure  comprehensive  PCI’s  are  conducted  prior  to  deploying  these  teams. 
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(15)  All  units  will  test  all  digital  systems  during  the  BCT  connectivity  exercise 
during  RSOI  (Script  TBP). 

(16)  Status  of  comms  systems  is  due  from  all  units  to  the  Brigade  S6  at  the  1BCT 
TOC  NLT  0300  and  1500  hrs  daily. 

(17)  MOPP  level  0  in  effect  25  1900  MAR  02. 

(18)  ADA  defense  warning  is  White. 

(19)  Weapons  control  Status  is  Hold. 

4.  SERVICE  SUPPORT. 

a.  Support  Concept-4th  FSB  with  attachments  provides  uninterrupted  CSS  to  the  1  BCT 
to  facilitate  SASO  operations  and  transition  immediately  to  combat  operations  dependant  on 
METT-T.  Support  to  civilian  personnel  is  limited  to  lifesaving  medical  treatment,  emergency 
water  and  food  only  in  the  most  extreme  of  circumstances. 

(1)  Support  Area  Locations 
-BSA  Location 

-ALT  BSA  Location 

-TFSA  s  are  located  in  Maneuver  BN  Headquarters  Base 

(2)  ARFOR  Priority  of  Support:  IBC,  1BCT,  1/325  PIR,  and  2BCT(upon  arrival) 

(3)  Priority  of  Support  (SASO)-  TF  1-22,  TF  1-66,  3-66  AR 

Maintenance-  BRT/MP  HMMWVs,  CSS,  Signal,  ENG  Equipment,  Mis, 

M2s,  FA,  and  ADA 

Movement-MSR  Patrols,  Resupply  Convoys,  ENG,  and  CBT  Vehicles 
Supplv-CL  I/Water,  CL  VIII,  CL  IX,  CL  III  and  CL  V 

(4)  Priority  of  Support(Combat)-  No  Change 

Maintenance-Mis,  M2s,  ENG,  ADA,  BRT,  Signal,  and  CSS 
Movement  to  FLOT-M1/M2,  FA,  ADA,  ENG,  Smoke,  and  CSS 
Movement  to  Rear-Medical,  empty  fuelers  and  cargo  vehicles, 
maintenance  evacuation. 

Priority  of  Supplv-CL  III,  CL  V,  CL  VIII,  CL  IX  and  CL  I/Water 

(5)  Unique  Support  requirements 

Man-Personnel  replacements  are  processed  through  BDE  ALCO  and 
pushed  forward  to  unit  LRP. 
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Arm-UBL  on  hand  in  each  unit.  Resupply  vicinity  the  BSA 
Fuel-Pushed  to  LRP  vicinity  the  BSA  by  the  7th  CSG 
Fix-CRTs  with  companies 

Move-No  transportation  units  available  to  augment 

Sustain:  Ration  cycle  (SASO)  A-M-A,  (Combat  first  5  days)  M-M-M, 

Bulk  water  available  at  BSA  collocated  with  HDC. 

(6)  Host  Nation  Support  -  The  ARFOR  contracting  officer  is  still  negotiating 

HNS. 

b.  Material  and  Services 

(1)  Mortuary  Affairs-One  MA  Team  from  the  7th  CSG  is  located  in  the  BSA  in 
the  HDC  area.  Units  are  responsible  for  the  transportation  of  remains  to  the  BSA.  Host  nation 
remains  are  not  processed  by  military  personnel  and  are  not  to  be  transported  in  US  Army 
vehicles 


c.  Medical  Evacuation  and  Hospitalization 

(1)  Casualty  Estimate 
-SASO:  32  Per  Day 
-Combat:  55  Per  Day 

(2)  Evacuation  (SASO)-  No  AXPs  active.  Unit’s  will  evacuate  injured  personnel 
to  the  nearest  Level  I  facility  located  with  a  Maneuver  BN  HQs,  Level  II  is  located  in  BSA. 

Level  III  treatment  facility,  134  CHS  is  located  at  the  Port. 

(3)  Evacuation  (Combat)-Two  AXPs  will  be  pushed  out  from  BSA  with  one 
initially  located  at  the  intersection  of  MSR  Pea  and  ASR  Apple  south  of  Kazar  City  and  the  other 
at  the  intersection  of  MSR  Squash  and  MSR  Brocolli. 

(4)  Evacuation  Air-  Attached  FST  has  three  MEDEVAC  helicopters  on  call.  BDE 
AL  F403  Call  Sign  Dust  Off.  Use  Army  9  line  MEDEVAC  Request.  Aerial  MEDVAC  is  used 
only  to  prevent  loss  of  life,  limb  or  eye-sight. 

d.  Personnel  Support-A  combined  personnel/finance  detachment  from  the  13th  PSB  is 
located  in  the  BSA  for  finance  and  administrative  actions. 


5.  COMMAND  AND  SIGNAL 


a.  Command. 

(1)  1BCT  TOC  will  be  located  at  (PV  179467). 

(2)  1BCT  ALOC  will  be  located  at  (PV  1 10390). 
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(3)  Command  succession.  TF  1-22  Cdr,  TF  1-66  Cdr,  3-66  Cdr. 
b.  Signal. 

(1)  Signal  Task  Organization.  See  Annex  A. 

(2)  ARFOR  signal  operating  instructions  (SOI)  edition  xx  is  in  effect. 

(3)  All  higher  headquarter  orders,  OPLANS,  FRAGOS,  etc.,  will  be  posted  on  the 
ARFOR  web  page  accessible  via  tactical  network.  1BCT  will  post  all  brigade  products  to  this 
web  site. 

(4)  Tactical  communications  systems  are  the  primary  means  of  communication.  (FM, 
FBCB2,  MSE,  ATCCS,  Single-channel  TACSAT).  The  host  nation  communications 
infrastructure  can  be  used,  but  tactical  communications  will  remain  the  primary  means  during  all 
phases. 
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Appendix  F  -  Checklist  Procedure  for  Executing  the  Java  Neural  Network  Simulation. 

Using  Java  Neural  Network  Simulator  (JNNS). 

•  The  type  of  NNS  being  used  here  is  a  Multi-Layer  Perceptron  (MLP). 

•  There  are  three  layers: 

o  Input. 

o  One  Hidden  with  30  nodes, 
o  Output. 

•  The  learning  algorithm  is  feed  forward  with  back  propagation  causing  the  error  value  to 
propagate  back  up  the  network. 

•  The  state  learning  parameters  are: 

o  Learning  rate  (Dmax). 
o  Level  of  error, 
o  Number  of  epochs. 

•  The  method  of  pruning  is  to  eliminate  the  non-contributing  parameters. 


1 .  To  start  JNNS  -  Execute  JavaNNS.bat  in  whatever  directory  that  JavaNNS-Win  is  set  up 
in,  for  example: 

C:\Documents  and  Settings\All  Users\JavaNNS-Win\JavaNNS.bat 
First  a  DOS  window  will  appear  and  then  the  JavaNNS  window. 

2.  Select  View  Network  to  display  the  network  sub  window. 

3.  Either  open  and  existing  file  with  the  extension  .net  or  create  a  new  network  data  file. 

4.  Use  Tools  -  Create  -  Layers  to  display  the  Create  Layers  sub  window  to  create  the  nodes 
in  the  layers.  The  layers  must  be  created  in  the  following  order: 

a.  Input. 

b.  Hidden. 

c.  Output. 

5.  In  the  Create  layers  sub  window  leave  the  width  and  height  set  to  1 .  Set  the  Top  left 
position  to  the  coordinate  on  the  network  window  where  the  new  node  will  be  placed 
when  created.  Set  Unit  type  to  either  Input,  Output  or  Hidden  for  the  layer  being  created. 
Leave  the  defaults  for  Activation  function  and  Output  function.  Set  the  Layer  number  to 
the  number  for  the  layer  that  the  node  is  being  created  for  (this  must  be  set  each  time  a 
node  is  created  as  the  default  will  always  be  wrong).  Use  the  following  layer  numbering 
assignments: 

a.  Input  -  1 . 

b.  Hidden  -  2.  Additional  hidden  layers  may  be  2,3,4  ,  etc  and  then  the  output  layer 
would  be  the  last  hidden  layer  plus  1 .  By  definition  only  one  hidden  layer  is 
required.  Some  complex  problems  can  be  better  solved  with  multiple  hidden 
layers.  There  is  no  guidance  on  the  number  of  hidden  layers  required.  The  rule  is 
try  one  first,  then  try  to  increase  the  number  of  nodes  in  the  hidden  layer.  If  the 
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network  still  does  not  train  effectively  then  try  increasing  the  number  of  hidden 
layers. 

c.  Output  -  3.  (One  greater  than  the  highest  hidden  layer). 

6.  Once  the  nodes  have  been  created  they  can  be  named  by  right  clicking  on  the  node  icon 
and  selecting  Edit  Units. . .  The  edit  units  sub  window  appears  where  parameters  for  the 
node  can  be  modified  including  the  default  name. 

7.  The  infonnation  displayed  above  and  below  the  node  icon  (Top  label  and  Base  label)  can 
be  set  by  selecting  the  Units  and  Links  tab  on  the  View  -  Display  Settings  pull  down 
menu 

8.  To  create  the  connections  between  all  the  nodes  select  Tools  -  Create  -  Connections.  In 
the  Create  links  sub  window  select  the  Connect  feed  -  forward  and  not  the  With  shortcut 
connections  check  boxes.  Click  the  Connect  button  to  execute  the  create  connections 
command.  Each  link  is  created  with  a  default  link  value  of  zero  “0”. 

9.  Next  need  to  load  the  data  set.  Go  to  the  file  open  menu  and  select  the  pattern  dataset 
with  the  extension  of  .pat.  Check  to  see  if  it  is  loaded  by  Tools  -  control  panel  and  the 
patterns  tab.  The  dataset  will  show  in  both  the  Training  set  and  the  Validation  set 
windows.  If  a  different  Validation  set  is  to  be  used  then  load  it  and  select  which  one  is 
the  training  and  which  one  is  the  validation  set. 

a.  The  .pat  file  was  previously  created  from  the  observation  data. 

b.  To  create  a  validation  file  make  a  random  selection  of  about  10-15%  of  the 
observation  data  and  save  it  into  a  separate  .pat  file.  Remove  the  validation 
patterns  from  the  remaining  training  patterns.  Renumber  the  pattern  sequence  for 
both  the  training  and  validation  pattern  files.  Read  them  both  in  and  indicate 
which  one  is  for  training  and  which  one  is  for  validation  under  the  Learning  tab  in 
the  control  panel. 

10.  Next  set  the  learning  parameters. 

a.  Select  tools  -  control  panel  and  the  learning  tab. 

b.  Set  learning  function  as  desired  -  Backpropagation  is  typical  when  there  are  no 
direct  links. 

c.  Set  parameter  n,  the  learning  rate,  typically  to  .  1  or  lower  but  greater  than  zero.  If 
set  too  high  then  it  learns  fast  then  it  could  be  stuck  in  local  maximums  and  could 
not  reach  the  true  optimal  value. 

d.  Set  parameter  dmax,  the  range  of  fluxuactions.  If  it  is  small  then  the  learning 
curve  becomes  smooth.  Typically  set  at  .1. 

e.  Set  cycles  (epochs)  to  500  as  a  start  to  test  the  network-  then  increase  to  1000  or 
even  10000.  Always  save  before  starting  training  in  case  it  hangs  up.  Setting  the 
cycles  small  gives  a  test  to  see  if  it  will  run.  This  works  in  conjunction  with 
repeated  uses  of  the  learn  all  button. 

f.  Set  steps  to  1  -  steps  are  the  number  of  cycles  before  the  connection  weights  are 
updated.  Always  set  to  1 . 

g.  Check  the  shuffle  check  box.  This  changes  the  presentation  order  of  the  data 
patterns  randomly  for  each  cycle  -  randomizes  the  data  order  for  each  cycle. 
Always  check  this  box.  This  may  not  be  used  for  datasets  that  may  contain  time 
series  data  sets.  Do  not  check  for  the  learning  function  Backprop  thru  time. 

h.  Turn  on  the  error  graph  with  the  selection  View  -  Error  Graph. 
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i.  Initialize  the  connection  weights  in  the  network  by  selecting  the  Init  button  in  the 
control  panel  This  assigns  random  initial  weights  to  each  connection.  The 
weights  will  show  on  the  network  diagram. 

j.  Start  the  learning  by  selecting  the  leam  all  button  in  the  control  panel.  Look  at 
the  error  graph  and  if  the  end  of  the  graph  is  still  going  down  then  repeat  the  learn 
all  selection  until  the  graph  starts  to  flatten  out.  Can  zoom  in  and  out  of  the  graph 
to  look  at  the  graph. 

k.  If  a  validation  dataset  is  used  then  there  will  be  2  curves  on  the  error  graph.  The 
training  set  may  continue  to  go  down  while  the  validation  set  will  turn  up.  At  the 
point  of  intersection  is  an  indicator  of  where  the  network  is  starting  to  overtrain. 

l.  A  validation  dataset  is  created  by  randomly  selecting  datapoints  from  the  training 
dataset.  The  validation  set  typically  has  about  10-20  %  of  the  original  dataset.  A 
validation  set  may  also  be  created  by  collecting  additional  data. 

m.  One  iteration  is  running  the  network  with  a  set  of  random  initial  values. 

Additional  iterations  can  be  run  with  different  initial  values.  Each  iteration  will 
show  as  a  separate  line  on  the  error  graph.  Can  also  change  n,  the  learning  rate,  to 
a  lower  number  such  as  .05,  and  also  dmax  to  a  larger  (as  in  .2)  or  smaller  (.01) 
values  and  try  again.  It  is  important  to  try  out  multiple  variations. 

n.  If  an  acceptable  level  of  error  cannot  be  achieved  through  multiple  iterations  then 
should  increase  the  number  of  nodes  in  the  hidden  layer  and  repeat  the  process. 

o.  Once  an  acceptable  level  of  error  has  been  achieved  with  a  flat  line  this  indicates 
possible  overtraining  then  the  network  is  ready  for  pruning.  A  better  method  is  to 
use  a  validation  dataset  and  see  where  the  graphs  cross  in  the  training  and 
validation  curves. 

i.  The  first  purpose  of  pruning  is  to  eliminate  the  overtrained  part  of  the 
network.  If  the  response  line  being  modeled  has  many  curves  then 
removing  some  of  the  nodes  may  cause  a  smoother  response  in  the 
regression  like  model.  This  is  analogous  to  removing  regressors  from  a 
regression  model  in  an  attempt  to  smooth  out  the  regression  model. 

ii.  The  second  purpose  is  to  eliminate  noncontributing  units  (nodes)  from  the 
trained  network.  The  first  purpose  is  automatically  achieved  by  doing  this 
second  purpose. 

1 1 .  Pruning. 

a.  Save  the  network  before  pruning  so  additional  pruning  iterations  can  be  run. 

b.  Select  pruning  tab  in  the  control  panel. 

c.  Select  Method:  Non  -  contributing  Units  as  the  typical  selection.  Can  also  use 
Magnitude  based  where  one  node  is  randomly  thrown  out  and  the  network  is 
tested  to  see  if  the  functions  call  still  occur. 

d.  Set  Maximum  error  increase  (%):  set  to  .001  -  real  low  so  that  the  network  is  not 
damaged.  If  set  too  high  then  most  of  the  connections  will  be  lost. 

e.  Set  Accepted  error:  set  to  .001  -  this  is  the  level  as  shown  on  the  error  graph. 

f.  Set  Cycles  for  retraining  -  start  at  100  -  this  is  the  number  of  epochs  that  will  be 
used  in  the  pruning.  Try  higher  numbers. 

g.  Set  Minimum  error  to  train  -  set  to  .001  - 

h.  Check  boxes: 

i.  Recreate  last  element  -  check  - 
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ii.  Refresh  display  -  check  -  the  display  will  show  animation  during  the 
pruning  process. 

iii.  Prune  input  units  -  check  -  prunes  input  units  /  nodes. 

iv.  Prune  hidden  units  -  typically  do  not  check  if  you  are  only  interested  in 
eliminating  input  nodes  only  to  see  their  effect. 

i.  Select  the  Prune  button.  This  step  cannot  be  reversed  so  be  sure  to  save  the 
unpruned  network  before  pruning  so  different  input  combinations  can  be  tested. 

Questions: 

Can  additional  nodes  be  added  after  the  network  has  already  been  built  and  the 
connections  made? 

Nodes  must  be  entered  in  the  order  input-hidden-output.  For  example  to  increase  nodes  in  the 
input  layer  first  delete  the  layers  hidden  and  output  and  add  them.  Then  recreate  the  hidden  and 
output  layers. 

Can  they  be  added  after  the  network  has  been  trained?  After  it  has  been  pruned? 

Yes,  using  the  same  rules  and  above 

Can  node  levels  and  other  information  be  changed  after  a  node  has  been  created? 

Yes  -  use  the  edit  units  box  and  change  the  Unit  type  and  layer  number. 

What  is  the  purpose  of  the  shortcut  connections  and  should  they  be  used? 

This  type  of  neural  network  model  is  called  multi-layer  perceptron.  Its  purpose  is  to  solve  XOR 
problems  is  where  two  binary  numbers  exist  with  the  rule  that  only  when  either  is  1  then  output 
is  1  -  the  exclusive  or  problem.  Before  the  XOR  problem  could  not  be  solved  hence  this  type  of 
network  model.  Direct  connections  of  input  to  output  help  to  solve  the  XOR  problem.  Generally 
do  not  use  this  option  because  it  causes  a  much  more  complex  network.  If  direct  connections 
are  used  use  the  quick  propagation  learning  algorithm.  If  direct  connections  are  not  used  then 
use  back  propagation  or  back  propagation  with  momentum  which  is  the  most  popular  one. 
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Appendix  G  -  Trained  JNNS  Output  Node  Values  For  TOC63  Simulation. 


Pattern 

Observed 

1-  PT1 1 

Observed 

2-  PT20 

Observed 

3-  PT  21 

Observed 

4-  ST  12 

Observed 

5-  ST  19 

Observed 

6- AT  21 

Calculated 

1-  PT1 1 

Calculated 

2-  PT20 

Calculated 

3-  PT  21 

Calculated 

4-  ST  12 

Calculated 

5-  ST  19 

Calculated 

6-  AT  21 

#1 

0 

0 

1 

0 

0 

0 

0.0022 

0.0011 

0.9984 

0.0045 

0.0010 

0.0066 

#2 

0 

0 

1 

0 

0 

0 

0.0031 

0.0156 

0.9829 

0.0001 

0.0151 

0.0063 

#3 

1 

0 

0 

1 

0 

1 

0.9846 

0.0363 

0.0041 

0.9642 

0.0265 

0.9821 

#4 

0 

0 

1 

0 

0 

0 

0.0124 

0.0035 

0.9919 

0.0008 

0.0129 

0.0051 

#5 

0 

0 

1 

0 

0 

0 

0.0027 

0.0326 

0.9801 

0.0011 

0.0073 

0.0025 

#6 

1 

0 

0 

1 

0 

1 

0.9912 

0.0020 

0.0160 

0.9754 

0.0138 

0.9909 

#7 

1 

0 

0 

1 

0 

1 

0.9940 

0.0014 

0.0173 

0.9890 

0.0064 

0.9895 

#8 

0 

0 

1 

0 

0 

0 

0.0062 

0.0021 

0.9961 

0.0008 

0.0060 

0.0061 

#9 

0 

0 

1 

0 

0 

0 

0.0069 

0.0062 

0.9940 

0.0153 

0.0008 

0.0052 

#10 

0 

0 

1 

0 

0 

0 

0.0055 

0.0013 

0.9994 

0.0006 

0.0057 

0.0066 

#11 

0 

0 

1 

0 

0 

0 

0.0074 

0.0047 

0.9887 

0.0030 

0.0030 

0.0100 

#12 

0 

0 

1 

0 

0 

0 

0.0063 

0.0053 

0.9908 

0.0068 

0.0014 

0.0066 

#13 

1 

0 

0 

1 

0 

1 

0.9934 

0.0020 

0.0108 

0.9654 

0.0383 

0.9921 

#14 

0 

0 

1 

0 

0 

0 

0.0043 

0.0041 

0.9966 

0.0005 

0.0060 

0.0040 

#15 

0 

0 

1 

0 

0 

0 

0.0099 

0.0025 

0.9925 

0.0217 

0.0022 

0.0159 

#16 

1 

0 

0 

1 

0 

1 

0.9851 

0.0024 

0 . 0228 

0.9820 

0.0032 

0.9836 

#17 

1 

0 

0 

1 

0 

1 

0.9952 

0.0012 

0.0139 

0.9931 

0.0059 

0.9894 

#18 

0 

0 

1 

0 

0 

0 

0.0028 

0.0261 

0.9854 

0.0009 

0.0054 

0.0016 

#19 

0 

0 

1 

0 

0 

0 

0.0061 

0.0254 

0.9709 

0.0025 

0.0115 

0.0078 

#20 

0 

1 

0 

0 

0 

0 

0.0204 

0.9403 

0.0616 

0.0012 

0.0164 

0.0111 

#21 

0 

1 

0 

0 

0 

0 

0.0170 

0 . 9414 

0.0380 

0.0040 

0.0038 

0.0267 

#22 

1 

0 

0 

1 

0 

1 

0.9873 

0.0044 

0.0144 

0.9579 

0.0543 

0.9908 

#23 

0 

0 

1 

0 

0 

0 

0.0022 

0.0014 

0.9983 

0.0063 

0.0010 

0.0058 

#24 

0 

0 

1 

0 

0 

0 

0.0037 

0.0143 

0.9719 

0.0042 

0.0010 

0.0038 

#25 

1 

0 

0 

1 

0 

1 

0.9827 

0.0178 

0.0018 

0.9960 

0.0018 

0.9882 

#26 

0 

0 

1 

0 

0 

0 

0.0063 

0.0057 

0.9828 

0.0007 

0.0096 

0.0082 

#27 

0 

0 

1 

0 

0 

0 

0.0091 

0.0033 

0.9963 

0.0006 

0.0216 

0.0036 

#28 

0 

0 

1 

0 

0 

0 

0.0152 

0.0078 

0.9792 

0.0115 

0.0038 

0.0199 

#29 

0 

1 

0 

0 

0 

0 

0.0199 

0.9534 

0.0434 

0.0277 

0.0008 

0.0186 

#30 

1 

0 

0 

1 

0 

1 

0.9905 

0.0018 

0.0083 

0.9895 

0.0064 

0.9918 
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Pattern 

Input  1- 
PT1 1 

Input  2- 
PT20 

Input  3- 
PT  21 

Input  4- 
ST  12 

Input  5- 
ST  19 

Input  6- 
AT  21 

Output  1- 
PT 1 1 

Output  2- 
PT20 

Output  3- 
PT  21 

Output  4- 
ST  12 

Output  5- 
ST  19 

Output  6- 
AT  21 

#31 

0 

0 

1 

0 

0 

0 

0.0125 

0.0007 

0.9959 

0.0255 

0.0011 

0.0082 

#32 

0 

1 

0 

0 

0 

0 

0.0189 

0 . 9744 

0.0194 

0.0021 

0.0108 

0.0257 

#33 

0 

0 

1 

0 

0 

0 

0.0040 

0.0076 

0.9916 

0.0014 

0.0034 

0.0020 

#34 

0 

1 

0 

0 

0 

0 

0.0137 

0.9538 

0.0415 

0.0228 

0.0006 

0.0140 

#35 

1 

0 

0 

1 

0 

1 

0.9880 

0.0173 

0.0014 

0.9926 

0.0072 

0.9946 

#36 

0 

0 

1 

0 

0 

0 

0.0035 

0.0065 

0.9901 

0.0053 

0.0013 

0.0090 

#37 

0 

0 

1 

0 

0 

0 

0.0056 

0.0025 

0.9973 

0.0017 

0.0029 

0.0055 

#38 

1 

0 

0 

1 

0 

1 

0.9925 

0.0060 

0.0042 

0.9964 

0.0034 

0.9903 

#39 

1 

0 

0 

0 

1 

1 

0.9709 

0 . 0114 

0.0169 

0.0632 

0.9267 

0.9763 

#40 

0 

0 

1 

0 

0 

0 

0.0035 

0.0167 

0.9809 

0.0017 

0.0032 

0 .0042 

#41 

0 

0 

1 

0 

0 

0 

0.0032 

0.0187 

0.9820 

0.0158 

0.0005 

0.0056 

#42 

1 

0 

0 

1 

0 

1 

0.9860 

0.0046 

0.0053 

0.9910 

0.0028 

0.9898 

#43 

0 

0 

1 

0 

0 

0 

0.0017 

0.0023 

0.9973 

0.0009 

0.0028 

0.0075 

#44 

0 

0 

1 

0 

0 

0 

0.0061 

0.0767 

0.9322 

0.0018 

0.0022 

0.0080 

#45 

0 

0 

1 

0 

0 

0 

0.0092 

0.0012 

0.9983 

0.0008 

0.0237 

0.0102 

#46 

0 

0 

1 

0 

0 

0 

0.0008 

0.0078 

0.9984 

0.0005 

0.0070 

0.0013 

#47 

0 

0 

1 

0 

0 

0 

0.0110 

0.0032 

0.9867 

0.0066 

0.0023 

0.0178 

#48 

1 

0 

0 

1 

0 

1 

0.9813 

0.0057 

0.0123 

0.9615 

0.0280 

0.9791 

#49 

0 

0 

1 

0 

0 

0 

0.0022 

0.0066 

0.9958 

0.0003 

0.0199 

0.0041 

#50 

1 

0 

0 

1 

0 

1 

0.9792 

0.0028 

0.0175 

0.9734 

0.0028 

0.9768 

#51 

0 

0 

1 

0 

0 

0 

0.0065 

0.0046 

0.9902 

0.0108 

0.0011 

0.0070 

#52 

0 

0 

1 

0 

0 

0 

0.0081 

0.0050 

0.9874 

0.0004 

0.0217 

0.0023 

#53 

0 

0 

1 

0 

0 

0 

0.0049 

0.0050 

0.9948 

0.0054 

0.0012 

0.0058 

#54 

1 

0 

0 

1 

0 

1 

0.9901 

0.0078 

0.0081 

0.9473 

0.0573 

0.9878 

#55 

1 

0 

0 

0 

1 

1 

0.9796 

0.0154 

0.0227 

0.0691 

0.9078 

0.9732 

#56 

0 

0 

1 

0 

0 

0 

0.0198 

0.0279 

0.9619 

0.0122 

0.0021 

0.0123 

#57 

1 

0 

0 

1 

0 

1 

0.9919 

0.0039 

0.0100 

0.9949 

0.0050 

0.9893 

#58 

1 

0 

0 

1 

0 

1 

0.9936 

0.0043 

0.0031 

0.9771 

0.0417 

0.9899 

#59 

1 

0 

0 

1 

0 

1 

0.9957 

0.0012 

0.0112 

0.9917 

0.0063 

0.9945 
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Appendix  H  -  SAS  Pearson  Correlation  Analysis  of  JNNS  Observed  to  Trained  DV  Data. 


options  pageno=l  formdlim="-"; 

TITLE  "10  Data  123  Correlation  Analysis'; 

DATA  iodatal23; 

INPUT  ITP1  ITP2  ITP3  ITP4  ITP5  ITP6  ITP7  ITP8  ITP9  ITP10 
ITP11  I TP 12  I TP 13  I TP 14  ITP15  ITP16  ITP17  ITP18  ITP19  ITP20 
ITP21  ITP22  ITP23  ITS1  ITS2  ITS3  ITS4  ITS5  ITS6  ITS7  ITS8  ITS9  ITS10 
ITS11  ITS12  ITS13  ITS14  ITS15  ITS16  ITS17  ITS18  ITS19  ITS20 
ITS21  ITS22  ITS23  ITA1  ITA2  ITA3  ITA4  ITA5  ITA6  ITA7  ITA8  ITA9  ITA10 
ITA11  ITA12  ITA13  ITA14  ITA15  ITA16  ITA17  ITA18  ITA19  ITA20 
ITA21  ITA22  ITA23  IM1  IM2  IM3  IM4  IM5  IM6  0TP1  0TP2  0TP3  0TP4  0TP5 
OTP 6  0TP7  0TP8  0TP9  OTPIO  0TP11  0TP12  0TP13  0TP14  0TP15  0TP16  0TP17 
0TP18  0TP19  OTP20  0TP21  OTP22  OTP23  0TS1  0TS2  0TS3  0TS4  0TS5  0TS6 
0TS7  0TS8  0TS9  OTS10  0TS11  0TS12  0TS13  0TS14  0TS15  0TS16  0TS17 
0TS18  0TS19  OTS20  0TS21  OTS22  OTS23  0TA1  0TA2  0TA3  0TA4  0TA5 
0TA6  0TA7  0TA8  0TA9  OTAIO  0TA11  0TA12  0TA13  0TA14  0TA15  0TA16 
0TA17  0TA18  0TA19  OTA20  0TA21  OTA22  OTA23  0M1  0M2  0M3  0M4  0M5  0M6; 


CARDS; 


0  0 

0  0  0 

0  0  0 

111 

0  0  0 

0  0  0 

0  0  0 

0  0  0 

0  0  0 

0  0  0 

0  0  0 

0  0  0 

0  10 

0  0  0  0 

0  0 

0  0  0 

0  0  0 

0  0  0 

0  0  0 

0  0  0 

0  0  0 

0  0  0 

1  0.0 

0.5  0. 

.5  0.0 

0.0  0. 

.0  0.008  0.014 

025 

0.012 

0.036 

0.000 

0.031 

0.020 

0.013 

0.984 

1.000 

0.968 

0.026 

0.004 

0.030 

0.029 

001 

0.002 

0.000 

0.001 

0.001 

0.000 

0.001 

0.001 

0.001 

0.000 

0.003 

0.005 

0.006 

0.011 

037 

0.002 

0.031 

0.020 

0.050 

0.000 

0.000 

0.978 

0.001 

0 . 017 

0.050 

0.000 

0.001 

0.000 

001 

0.010 

0.002 

0.000 

0.012 

0.012 

0.025 

0.002 

0.000 

0.007 

0.000 

0.004 

0.012 

0.028 

002 

0 . 024 

0.001 

0.000 

0.002 

0.016 

0.000 

0.007 

0.003 

0.001 

0.963 

0 . 017 

0 .464 

0 . 471 

0.038  0.017  0.001 


00000000001000000000000000000000001000000000 
0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  1  0  0  0.0  0.4  0.0  0.0  0.0  0.0  0.011  0.001 
0.001  0.010  0.002  0.000  0.017  0.008  0.011  0.011  0.989  0.043  0.003  0.011  0.001  0.001 

0.002  0.002  0.000  0.007  0.006  0.000  0.001  0.002  0.001  0.001  0.000  0.001  0.006  0.002 

0.003  0.000  0.005  0.003  0.998  0.000  0.005  0.009  0.001  0.001  0.002  0.002  0.042  0.001 

0.001  0.000  0.001  0.006  0.001  0.001  0.003  0.001  0.006  0.003  0.006  0.001  0.001  0.002 

0.003  0.000  0.000  0.005  0.003  0.002  0.007  0.002  0.979  0.000  0.018  0.000  0.432  0.008 

0.001  0.039  0.000 


*  IO  Correlation  Analysis; 

*  Run  corr  on  the  input  vs  output  variables  to  get  simple  statistics  table  showing  the 
sum  of  each  variable; 

*  which  will  indicate  the  number  of  observations  for  that  variable; 

*  • 


*  Running  this  short  correlation  when  SAS  first  starts  up  is  also  a  good  way 
database  loaded  and  ; 

*  available  for  other  analyses; 
options  pageno=l  f ormdlim="-" ; 
proc  corr  data=iodatal23; 

TITLE  "Observed  to  Trained  NNS  Output  for  Exercises  1,2,3"; 

*  var  itpl  itp2  otpl  otp2; 

var  itp2  otp2  itp3  otp3  itp4  otp4  itp5  otp5  itp7  otp7  itp8  otp8  itplO  otplO 
otpll 

itpl2  otpl2  itpl5  otpl5  itp22  otp22  itp23  otp23  itsl  otsl  its2  ots2  its5  ots5 
itsll  otsll  itsl2  otsl2  itsl5  otsl5  itsl8  otsl8  itsl9  otsl9  its20  ots20  its22 
its23 

ots23  ita2  ota2  ita3  ota3  ita7  ota7  italO  otalO  itall  otall  ital2  otal2  ital8 
ital9 

otal9  ita21  ota21  ita23  ota23  iml  oml  im2  om2  im3  om3  im4  om4  im5  om5  ; 


to  get  the 


itpll 

its8  ots8 
ots22 

otal8 
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Output: 


78  Variables: 


Observed  to  Trained  NNS  Output  for  Exercises  1,2,3  1 

07:43  Wednesday,  January  15,  2003 

The  CORR  Procedure 


ITP2 

OTP2 

ITP3 

OTP3 

ITP4 

0TP4 

ITP5 

OTP5 

ITP7 

0TP7 

ITP8 

0TP8 

ITP10 

0TP10 

ITP1 1 

OTP1 1 

ITP12 

OTP12 

ITP15 

OTP15 

ITP22 

OTP22 

ITP23 

OTP23 

ITS1 

OTS1 

ITS2 

OTS2 

ITS5 

OTS5 

ITS8 

OTS8 

ITS1 1 

OTS11 

ITS12 

OTS12 

ITS15 

OTS15 

ITS18 

0TS18 

ITS19 

0TS19 

ITS20 

0TS20 

ITS22 

OTS22 

ITS23 

OTS23 

ITA2 

OTA2 

ITA3 

OTA3 

ITA7 

0TA7 

ITA10 

0TA10 

ITA1 1 

OTA1 1 

ITA12 

OTA12 

ITA18 

0TA18 

ITA19 

OTA19 

ITA21 

OTA21 

ITA23 

OTA23 

IM1 

0M1 

IM2 

0M2 

IM3 

0M3 

IM4 

0M4 

IM5 

0M5 
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Simple  Statistics 


Variable 

N 

Mean 

Std  Dev 

Sum 

Minimum 

Maximum 

ITP2 

139 

0.00719 

0.08482 

1  .00000 

0 

1 .00000 

0TP2 

139 

0.00553 

0.00918 

0.76900 

0 

0.05000 

ITP3 

139 

0.04317 

0.20396 

6.00000 

0 

1 .00000 

0TP3 

139 

0.04994 

0.19439 

6.94200 

0 

0.97300 

ITP4 

139 

0.00719 

0.08482 

1  .00000 

0 

1 .00000 

0TP4 

139 

0.01293 

0.07550 

1  .79700 

0 

0.88400 

ITP5 

139 

0.04317 

0.20396 

6.00000 

0 

1 .00000 

0TP5 

139 

0.04859 

0.19338 

6.75400 

0 

0.96200 

ITP7 

139 

0.05755 

0.23374 

8.00000 

0 

1 .00000 

0TP7 

139 

0.06871 

0.22446 

9.55100 

0 

0.99400 

ITP8 

139 

0.01439 

0.11952 

2.00000 

0 

1 .00000 

0TP8 

139 

0.02480 

0.11118 

3.44700 

0.00100 

0.94500 

ITP10 

139 

0.10791 

0.31139 

15.00000 

0 

1 .00000 

0TP10 

139 

0.11249 

0.30231 

15.63600 

0 

0.99900 

ITP1 1 

139 

0.43165 

0.49710 

60.00000 

0 

1 .00000 

0TP1 1 

139 

0.43596 

0.47717 

60.59900 

0 

1 .00000 

ITP12 

139 

0.08633 

0.28187 

12.00000 

0 

1 .00000 

0TP12 

139 

0.09551 

0.26966 

13.27600 

0 

0.99600 

ITP15 

139 

0.09353 

0.29222 

13.00000 

0 

1 .00000 

0TP15 

139 

0.09894 

0.28060 

13.75200 

0 

0.99700 

ITP22 

139 

0.05036 

0.21948 

7.00000 

0 

1 .00000 

0TP22 

139 

0.05629 

0.21142 

7.82500 

0 

0.99800 

ITP23 

139 

0.10791 

0.31139 

15.00000 

0 

1 .00000 

0TP23 

139 

0.11085 

0.29980 

15.40800 

0 

0.99800 

ITS1 

139 

0.00719 

0.08482 

1  .00000 

0 

1 .00000 

0TS1 

139 

0.01226 

0.07887 

1 .70400 

0 

0.92800 

ITS2 

139 

0.04317 

0.20396 

6.00000 

0 

1 .00000 

0TS2 

139 

0.05004 

0.19682 

6.95500 

0 

0.99000 

ITS5 

139 

0.04317 

0.20396 

6.00000 

0 

1 .00000 

0TS5 

139 

0.04932 

0.19380 

6.85500 

0 

0.97900 

ITS8 

139 

0.00719 

0.08482 

1  .00000 

0 

1 .00000 

0TS8 

139 

0.00738 

0.01001 

1 .02600 

0 

0.05000 

ITS11 

139 

0.03597 

0.18689 

5.00000 

0 

1 .00000 

0TS11 

139 

0.04899 

0.17669 

6.81000 

0 

0.96800 

ITS12 

139 

0.20144 

0.40253 

28.00000 

0 

1 .00000 

0TS12 

139 

0.20658 

0.38491 

28.71400 

0 

0.99800 

ITS15 

139 

0.02878 

0.16778 

4.00000 

0 

1 .00000 

0TS15 

139 

0.03504 

0.16019 

4.87100 

0 

0.97800 

ITS18 

139 

0.10072 

0.30205 

14.00000 

0 

1 .00000 

0TS18 

139 

0.10267 

0.28929 

14.27100 

0 

0.98100 

ITS19 

139 

0.08633 

0.28187 

12.00000 

0 

1 .00000 

0TS19 

139 

0.09393 

0.26919 

13.05600 

0 

0.99200 

ITS20 

139 

0.01439 

0.11952 

2.00000 

0 

1 .00000 

0TS20 

139 

0.02371 

0.10744 

3.29600 

0 

0.91900 

ITS22 

139 

0.01439 

0.11952 

2.00000 

0 

1 .00000 

0TS22 

139 

0.02383 

0.11025 

3.31200 

0 

0.92800 

ITS23 

139 

0.12950 

0.33696 

18.00000 

0 

1 .00000 

0TS23 

139 

0.13570 

0.32150 

18.86200 

0 

0.99000 

ITA2 

139 

0.00719 

0.08482 

1  .00000 

0 

1 .00000 

0TA2 

139 

0.01717 

0.08006 

2.38600 

0 

0.94300 

ITA3 

139 

0.08633 

0.28187 

12.00000 

0 

1 .00000 

0TA3 

139 

0.09373 

0.27009 

13.02800 

0 

0.99100 

ITA7 

139 

0.04317 

0.20396 

6.00000 

0 

1 .00000 
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Simple  Statistics 


Variable 

N 

Mean 

Std  Dev 

Sum 

Minimum 

Maximum 

0TA7 

139 

0.05236 

0.19442 

7.27800 

0 

0.97800 

ITA10 

139 

0.01439 

0.11952 

2.00000 

0 

1  .00000 

0TA10 

139 

0.02231 

0.11555 

3.10100 

0 

0.98700 

ITA1 1 

139 

0.05036 

0.21948 

7.00000 

0 

1  .00000 

0TA1 1 

139 

0.05843 

0.20806 

8.12200 

0 

0.97800 

ITA12 

139 

0.02158 

0.14584 

3.00000 

0 

1  .00000 

0TA12 

139 

0.02882 

0.13936 

4.00600 

0 

0.96900 

ITA18 

139 

0.05036 

0.21948 

7.00000 

0 

1  .00000 

0TA18 

139 

0.05845 

0.20810 

8.12500 

0 

0.97100 

ITA19 

139 

0.02158 

0.14584 

3.00000 

0 

1  .00000 

0TA19 

139 

0.03160 

0.13681 

4.39300 

0 

0.95000 

ITA21 

139 

0.15827 

0.36632 

22.00000 

0 

1  .00000 

0TA21 

139 

0.16578 

0.35213 

23.04400 

0 

0.99400 

ITA23 

139 

0.19424 

0.39705 

27.00000 

0 

1  .00000 

0TA23 

139 

0.19943 

0.38212 

27.72100 

0 

0.99800 

IM1 

139 

0.16835 

0.3321 1 

23.40000 

0 

1  .00000 

0M1 

139 

0.17009 

0.32012 

23.64300 

0 

0.99200 

IM2 

139 

0.22158 

0.20913 

30.80000 

0 

0.60000 

0M2 

139 

0.23159 

0.19982 

32.19100 

0.00200 

0.64400 

I  M3 

139 

0.08417 

0.22320 

11 .70000 

0 

0.85000 

0M3 

139 

0.09353 

0.21926 

13.00000 

0 

0.89800 

IM4 

139 

0.00576 

0.04781 

0.80000 

0 

0.40000 

0M4 

139 

0.01485 

0.04307 

2.06400 

0 

0.36300 

IM5 

139 

0.07050 

0.21551 

9.80000 

0 

0.90000 

0M5 

139 

0.07935 

0.20647 

11 .02900 

0 

0.92800 
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Observed  to  Trained  NNS  Output  for  Exercises  1,2,3  3 

07:43  Wednesday,  January  15,  2003 

The  CORR  Procedure 

Pearson  Correlation  Coefficients,  N  =  139 
Prob  >  | r |  under  HO:  Rho=0 


ITP2 

OTP2 

ITP3 

OTP3 

ITP4 

0TP4 

ITP5 

OTP5 

ITP7 

ITP2 

1  .00000 

-0.05150 

0.40078 

0.39557 

-0.00725 

-0.00897 

-0.01808 

-0.01351 

-0.02104 

0.5471 

<.0001 

<.0001 

0.9325 

0.9165 

0.8327 

0.8745 

0.8058 

OTP2 

-0.05150 

1 .00000 

0.25475 

0.25682 

-0.04219 

-0.04770 

0.26637 

0.28439 

0.32342 

0.5471 

0.0025 

0.0023 

0.6219 

0.5771 

0.0015 

0.0007 

0.0001 

ITP3 

0.40078 

0.25475 

1 .00000 

0.99832 

-0.01808 

-0.02615 

0.47744 

0.48325 

-0.05249 

<.0001 

0.0025 

<.0001 

0.8327 

0.7599 

<.0001 

<.0001 

0.5394 

OTP3 

0.39557 

0.25682 

0.99832 

1 .00000 

-0.02151 

-0.03016 

0.47562 

0.48256 

-0.05846 

<.0001 

0.0023 

<.0001 

0.8016 

0.7245 

<.0001 

<.0001 

0.4943 

ITP4 

-0.00725 

-0.04219 

-0.01808 

-0.02151 

1 .00000 

0.98570 

-0.01808 

0.00283 

-0.02104 

0.9325 

0.6219 

0.8327 

0.8016 

<.0001 

0.8327 

0.9736 

0.8058 

0TP4 

-0.00897 

-0.04770 

-0.02615 

-0.03016 

0.98570 

1 .00000 

0.03408 

0.05530 

-0.01126 

0.9165 

0.5771 

0.7599 

0.7245 

<.0001 

0.6904 

0.5179 

0.8953 

ITP5 

-0.01808 

0.26637 

0.47744 

0.47562 

-0.01808 

0.03408 

1 .00000 

0.99784 

-0.05249 

0.8327 

0.0015 

<■0001 

<.0001 

0.8327 

0.6904 

<.0001 

0.5394 

OTP5 

-0.01351 

0.28439 

0.48325 

0.48256 

0.00283 

0.05530 

0.99784 

1 .00000 

-0.05125 

0.8745 

0.0007 

<.0001 

<.0001 

0.9736 

0.5179 

<.0001 

0.5490 

ITP7 

-0.02104 

0.32342 

-0.05249 

-0.05846 

-0.02104 

-0.01126 

-0.05249 

-0.05125 

1 .00000 

0.8058 

0.0001 

0.5394 

0.4943 

0.8058 

0.8953 

0.5394 

0.5490 

0TP7 

-0.00750 

0.33248 

-0.05528 

-0.06153 

-0.01435 

-0.00278 

-0.05069 

-0.04843 

0.99782 

0.9301 

<.0001 

0.5180 

0.4718 

0.8668 

0.9741 

0.5534 

0.5713 

<.0001 

ITP8 

-0.01029 

-0.05328 

-0.02566 

-0.02180 

-0.01029 

0.03465 

-0.02566 

-0.02451 

0.48893 

0.9043 

0.5333 

0.7643 

0.7990 

0 . 9043 

0.6855 

0 . 7643 

0.7746 

<.0001 

0TP8 

0.01860 

-0.05224 

-0.02454 

-0.02049 

-0.01598 

0.02968 

-0.04020 

-0.03806 

0.48285 

0.8280 

0.5414 

0 . 7743 

0.8108 

0.8519 

0.7287 

0.6385 

0.6564 

<.0001 

ITP10 

-0.02961 

0.37279 

-0.07387 

-0.05221 

-0.02961 

-0.02124 

-0.07387 

-0.05088 

-0.08595 

0.7294 

<.0001 

0.3874 

0.5416 

0.7294 

0.8040 

0.3874 

0.5519 

0.3144 

0TP10 

-0.03066 

0.38401 

-0.05664 

-0.03495 

-0.03094 

-0.02144 

-0.06357 

-0.04053 

-0.07896 

0.7201 

<.0001 

0.5078 

0.6829 

0.7177 

0.8022 

0.4572 

0.6357 

0.3555 

ITP1 1 

0.09768 

0.00486 

0.02931 

0.03685 

-0.07419 

-0.04975 

-0.18510 

-0.16481 

-0.09063 

0.2526 

0.9548 

0.7320 

0.6667 

0.3854 

0.5608 

0.0291 

0.0525 

0.2887 
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Observed  to  Trained  NNS  Output  for  Exercises  1,2,3  4 

07:43  Wednesday,  January  15,  2003 

The  CORR  Procedure 

Pearson  Correlation  Coefficients,  N  =  139 
Prob  >  | r |  under  HO:  Rho=0 


0TP7 

ITP8 

0TP8 

ITP10 

0TP10 

ITP1 1 

OTP1 1 

ITP12 

0TP12 

ITP2 

-0.00750 

-0.01029 

0.01860 

-0.02961 

-0.03066 

0.09768 

0.09203 

-0.02617 

-0.02709 

0.9301 

0 . 9043 

0.8280 

0.7294 

0.7201 

0.2526 

0.2812 

0.7598 

0.7516 

OTP2 

0.33248 

-0.05328 

-0.05224 

0.37279 

0.38401 

0.00486 

0.00611 

-0.0431 1 

-0.03612 

<.0001 

0 . 5333 

0.5414 

<.0001 

<.0001 

0.9548 

0.9431 

0.6143 

0.6729 

ITP3 

-0.05528 

-0.02566 

-0.02454 

-0.07387 

-0.05664 

0.02931 

0.02823 

-0.06529 

-0.05126 

0.5180 

0 . 7643 

0.7743 

0.3874 

0.5078 

0.7320 

0.7414 

0.4451 

0.5490 

0TP3 

-0.06153 

-0.02180 

-0.02049 

-0.05221 

-0.03495 

0.03685 

0.03577 

-0.04170 

-0.02768 

0.4718 

0.7990 

0.8108 

0.5416 

0.6829 

0.6667 

0.6759 

0.6260 

0.7463 

ITP4 

-0.01435 

-0.01029 

-0.01598 

-0.02961 

-0.03094 

-0.07419 

-0.06910 

-0.02617 

-0.02994 

0.8668 

0 . 9043 

0.8519 

0.7294 

0.7177 

0.3854 

0.4189 

0.7598 

0.7264 

0TP4 

-0.00278 

0.03465 

0.02968 

-0.02124 

-0.02144 

-0.04975 

-0.04418 

-0.03614 

-0.04023 

0.9741 

0.6855 

0.7287 

0.8040 

0.8022 

0.5608 

0.6056 

0.6727 

0.6382 

ITP5 

-0.05069 

-0.02566 

-0.04020 

-0.07387 

-0.06357 

-0.18510 

-0.17942 

-0.06529 

-0.06193 

0.5534 

0 . 7643 

0.6385 

0.3874 

0.4572 

0.0291 

0.0346 

0.4451 

0.4689 

OTP5 

-0.04843 

-0.02451 

-0.03806 

-0.05088 

-0.04053 

-0.16481 

-0.15918 

-0.04973 

-0.04601 

0.5713 

0.7746 

0.6564 

0.5519 

0.6357 

0.0525 

0.0613 

0.5610 

0.5907 

ITP7 

0.99782 

0.48893 

0.48285 

-0.08595 

-0.07896 

-0.09063 

-0.08769 

-0.07596 

-0.07175 

<.0001 

<.0001 

<.0001 

0.3144 

0.3555 

0.2887 

0.3047 

0.3741 

0.4013 

0TP7 

1  .00000 

0.47503 

0.47245 

-0.06404 

-0.05762 

-0.06350 

-0.06008 

-0.06191 

-0.05742 

<.0001 

<.0001 

0.4539 

0.5005 

0.4577 

0.4824 

0.4690 

0.5019 

ITP8 

0.47503 

1 .00000 

0.99383 

-0.04202 

-0.02567 

0.13864 

0.14321 

-0.03714 

-0.02789 

<.0001 

<.0001 

0.6233 

0.7642 

0.1036 

0.0926 

0.6642 

0 . 7445 

OTP8 

0.47245 

0.99383 

1 .00000 

0.00273 

0.01751 

0.16469 

0.16978 

-0.03852 

-0.02867 

<.0001 

<.0001 

0.9746 

0.8379 

0.0527 

0.0457 

0.6526 

0.7376 

ITP10 

-0.06404 

-0.04202 

0.00273 

1 .00000 

0.99906 

0.39909 

0.40622 

0.22333 

0.22699 

0.4539 

0.6233 

0.9746 

<.0001 

<.0001 

<.0001 

0.0082 

0.0072 

0TP10 

-0.05762 

-0.02567 

0.01751 

0.99906 

1 .00000 

0.40421 

0.41139 

0.22656 

0.23082 

0.5005 

0.7642 

0.8379 

<.0001 

<.0001 

<.0001 

0.0073 

0.0063 

ITP1 1 

-0.06350 

0.13864 

0.16469 

0.39909 

0.40421 

1 .00000 

0.99938 

0.35272 

0.37205 

0.4577 

0.1036 

0.0527 

<.0001 

<.0001 

<.0001 

<.0001 

<.0001 
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Observed  to  Trained  NNS  Output  for  Exercises  1,2,3  5 

07:43  Wednesday,  January  15,  2003 

The  CORR  Procedure 

Pearson  Correlation  Coefficients,  N  =  139 
Prob  >  | r |  under  HO:  Rho=0 


ITP15 

0TP15 

ITP22 

0TP22 

ITP23 

0TP23 

ITS1 

OTS1 

ITS2 

ITP2 

-0.02734 

-0.02951 

-0.01960 

-0.01952 

-0.02961 

-0.01763 

-0.00725 

-0.01111 

-0.01808 

0.7493 

0.7302 

0.8188 

0.8196 

0.7294 

0.8368 

0.9325 

0.8967 

0.8327 

OTP2 

0.38661 

0.4041 1 

0.09092 

0.09377 

-0.00757 

0.00247 

0.16261 

0.17476 

0.40186 

<.0001 

<.0001 

0.2871 

0.2722 

0.9295 

0.9770 

0.0558 

0.0396 

<.0001 

ITP3 

-0.06822 

-0.05921 

-0.04891 

-0.05189 

-0.07387 

-0.05381 

-0.01808 

0.00696 

-0.0451 1 

0.4249 

0.4887 

0.5675 

0.5441 

0.3874 

0.5292 

0.8327 

0.9352 

0.5980 

OTP3 

-0.06267 

-0.05357 

-0.05649 

-0.05975 

-0.07005 

-0.04972 

-0.01492 

0.01022 

-0.05422 

0.4636 

0.5311 

0.5089 

0.4847 

0.4126 

0.5611 

0.8616 

0.9050 

0.5261 

ITP4 

-0.02734 

-0.01764 

-0.01960 

-0.02275 

-0.02961 

-0.03073 

-0.00725 

-0.00570 

-0.01808 

0.7493 

0.8367 

0.8188 

0.7904 

0.7294 

0.7195 

0.9325 

0.9469 

0.8327 

0TP4 

0.00622 

0.01526 

-0.03564 

-0.03942 

-0.04282 

-0.04346 

-0.00558 

-0.00285 

-0.03321 

0.9421 

0.8585 

0.6770 

0.6449 

0.6167 

0.6115 

0.9481 

0.9735 

0.6980 

ITP5 

-0.06822 

-0.05389 

-0.04891 

-0.05508 

-0.07387 

-0.06069 

-0.01808 

-0.01241 

-0.0451 1 

0.4249 

0.5287 

0.5675 

0.5196 

0.3874 

0.4779 

0.8327 

0.8847 

0.5980 

OTP5 

-0.05664 

-0.04166 

-0.05756 

-0.06406 

-0.07387 

-0.06031 

-0.02102 

-0.01454 

-0.04437 

0.5078 

0.6263 

0.5009 

0.4538 

0.3875 

0.4806 

0.8059 

0.8651 

0.6040 

ITP7 

-0.07938 

-0.07485 

-0.05691 

-0.05372 

-0.08595 

-0.08260 

-0.02104 

-0.03422 

0.85949 

0.3530 

0.3812 

0.5058 

0.5299 

0.3144 

0.3337 

0.8058 

0.6892 

<.0001 

0TP7 

-0.06996 

-0.06522 

-0.05634 

-0.05338 

-0.09649 

-0.09339 

-0.01892 

-0.03176 

0.86513 

0.4131 

0.4456 

0.5101 

0.5326 

0.2585 

0.2742 

0.8250 

0.7106 

<.0001 

ITP8 

-0.03881 

-0.03843 

-0.02782 

-0.03114 

-0.04202 

-0.04261 

-0.01029 

-0.01270 

-0.02566 

0.6501 

0.6533 

0.7451 

0.7159 

0.6233 

0.6184 

0 . 9043 

0.8821 

0.7643 

0TP8 

-0.02908 

-0.02823 

-0.03433 

-0.03780 

-0.05776 

-0.05850 

0.00630 

0 . 00443 

-0.02901 

0.7340 

0.7415 

0.6883 

0.6587 

0.4994 

0.4939 

0.9413 

0.9587 

0.7346 

ITP10 

0.60499 

0.61974 

-0.08009 

-0.08524 

-0.12097 

-0.12286 

-0.02961 

-0.04599 

-0.07387 

<.0001 

<.0001 

0.3486 

0.3184 

0.1560 

0.1496 

0.7294 

0.5908 

0.3874 

0TP10 

0.61657 

0.63150 

-0.08578 

-0.09107 

-0.11388 

-0.11547 

-0.03179 

-0.04796 

-0.07544 

<.0001 

<.0001 

0.3154 

0.2863 

0.1819 

0.1759 

0.7103 

0.5750 

0.3774 

ITP1 1 

0.16903 

0.16322 

-0.20069 

-0.22076 

-0.30311 

-0.30084 

-0.07419 

-0.08678 

-0.18510 

0.0467 

0.0549 

0.0178 

0.0090 

0.0003 

0.0003 

0.3854 

0.3097 

0.0291 
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Observed  to  Trained  NNS  Output  for  Exercises  1,2,3  6 

07:43  Wednesday,  January  15,  2003 

The  CORR  Procedure 

Pearson  Correlation  Coefficients,  N  =  139 
Prob  >  | r |  under  HO:  Rho=0 


OTS2 

ITS5 

OTS5 

ITS8 

0TS8 

ITS1 1 

0TS11 

ITS12 

0TS12 

ITP2 

-0.01825 

-0.01808 

-0.02174 

-0.00725 

-0.00325 

-0.01644 

-0.02320 

-0.04275 

-0.03941 

0.8312 

0.8327 

0.7995 

0.9325 

0.9697 

0.8476 

0.7863 

0.6173 

0.6450 

OTP2 

0.40547 

-0.07043 

-0.07346 

-0.03288 

0.24461 

0.44504 

0.46239 

-0.11163 

-0.11793 

<.0001 

0.4100 

0.3901 

0.7008 

0.0037 

<.0001 

<.0001 

0.1908 

0.1668 

ITP3 

-0.04968 

-0.0451 1 

-0.05259 

-0.01808 

0.40703 

0.52926 

0.53223 

-0.10668 

-0.10517 

0.5614 

0.5980 

0.5386 

0.8327 

<.0001 

<.0001 

<.0001 

0.2113 

0.2179 

OTP3 

-0.05921 

-0.05038 

-0.05832 

-0.01843 

0.42105 

0.52643 

0.53092 

-0.11876 

-0.11722 

0.4887 

0.5559 

0.4953 

0.8295 

<.0001 

<.0001 

<.0001 

0.1638 

0.1694 

ITP4 

-0.01347 

0.40078 

0.39748 

-0.00725 

-0.04592 

-0.01644 

-0.00725 

-0.04275 

-0.04186 

0.8749 

<.0001 

<.0001 

0.9325 

0.5914 

0.8476 

0.9325 

0.6173 

0.6247 

0TP4 

-0.02732 

0.45948 

0.46063 

0.09287 

-0.06368 

-0.00341 

0.00622 

-0.02837 

-0.02631 

0.7496 

<.0001 

<.0001 

0.2769 

0.4564 

0.9682 

0.9420 

0.7402 

0.7585 

ITP5 

-0.04119 

-0.0451 1 

-0.03280 

0.40078 

0.31478 

0.52926 

0.54912 

-0.10668 

-0.10822 

0.6302 

0.5980 

0.7015 

<■0001 

0.0002 

<.0001 

<.0001 

0.2113 

0.2048 

OTP5 

-0.04024 

-0.03262 

-0.02106 

0.39867 

0.33548 

0.53133 

0.55314 

-0.11036 

-0.11165 

0.6381 

0.7031 

0.8056 

<.0001 

<.0001 

<.0001 

<.0001 

0.1959 

0.1907 

ITP7 

0.85715 

-0.05249 

-0.05591 

-0.02104 

-0.11781 

-0.04774 

-0.06315 

0.02992 

0.02492 

<.0001 

0.5394 

0.5133 

0.8058 

0.1672 

0.5768 

0.4602 

0.7266 

0.7709 

0TP7 

0.86334 

-0.05481 

-0.05733 

-0.01740 

-0.11295 

-0.05106 

-0.06678 

0.03786 

0.03352 

<.0001 

0.5216 

0.5026 

0.8389 

0.1855 

0.5506 

0.4348 

0.6581 

0.6952 

ITP8 

-0.02805 

-0.02566 

-0.03054 

-0.01029 

-0.03490 

-0.02334 

-0.03294 

0.24057 

0.23909 

0.7430 

0.7643 

0.7211 

0 . 9043 

0.6834 

0.7851 

0.7003 

0 . 0043 

0.0046 

0TP8 

-0.0321 1 

-0.04275 

-0.04763 

-0.01521 

-0.04860 

-0.03871 

-0.04955 

0.25610 

0.25514 

0.7074 

0.6173 

0.5776 

0.8589 

0.5699 

0.6510 

0.5624 

0.0023 

0.0024 

ITP10 

-0.08082 

-0.07387 

-0.07442 

-0.02961 

0.08665 

-0.06718 

-0.05003 

-0.05906 

-0.05759 

0.3443 

0.3874 

0.3840 

0.7294 

0.3105 

0.4320 

0.5586 

0.4898 

0.5007 

0TP10 

-0.08227 

-0.06992 

-0.07090 

-0.03151 

0.09610 

-0.05200 

-0.03418 

-0.05894 

-0.05750 

0.3357 

0.4134 

0.4069 

0.7127 

0.2604 

0.5432 

0.6896 

0.4907 

0.5014 

ITP11 

-0.18384 

0.02931 

0.03400 

-0.07419 

0.18362 

-0.16834 

-0.16934 

0.43145 

0.44024 

0.0303 

0.7320 

0.6911 

0.3854 

0.0305 

0.0476 

0.0463 

<.0001 

<.0001 
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Observed  to  Trained  NNS  Output  for  Exercises  1,2,3  7 

07:43  Wednesday,  January  15,  2003 

The  CORR  Procedure 

Pearson  Correlation  Coefficients,  N  =  139 
Prob  >  | r |  under  HO:  Rho=0 


ITS15 

OTS15 

ITS18 

0TS18 

ITS19 

0TS19 

ITS20 

0TS20 

ITS22 

ITP2 

-0.01465 

-0.01709 

0.25436 

0.25053 

-0.02617 

-0.02917 

-0.01029 

-0.00454 

-0.01029 

0.8641 

0.8417 

0.0025 

0.0029 

0.7598 

0.7332 

0 . 9043 

0.9577 

0 . 9043 

OTP2 

0.04645 

0.07270 

0.25239 

0.25693 

-0.14956 

-0.15260 

0.03261 

0.04463 

0.01279 

0.5871 

0.3950 

0.0027 

0.0023 

0.0789 

0.0729 

0.7032 

0.6019 

0.8813 

ITP3 

-0.03656 

-0.01869 

0.28179 

0.28246 

-0.06529 

-0.05564 

-0.02566 

-0.03944 

-0.02566 

0.6692 

0.8272 

0.0008 

0.0008 

0.4451 

0.5153 

0 . 7643 

0.6448 

0.7643 

OTP3 

-0.01683 

0.00124 

0.29556 

0.29705 

-0.05916 

-0.04984 

-0.02554 

-0.04066 

-0.02772 

0.8441 

0.9885 

0.0004 

0.0004 

0.4891 

0.5601 

0.7654 

0.6346 

0.7460 

ITP4 

-0.01465 

-0.00322 

-0.02849 

-0.02766 

-0.02617 

-0.02949 

-0.01029 

0.01931 

-0.01029 

0.8641 

0.9700 

0.7392 

0.7465 

0.7598 

0.7304 

0 . 9043 

0.8215 

0 . 9043 

0TP4 

-0.00956 

0.00679 

-0.03559 

-0.03467 

-0.04738 

-0.05147 

0.02180 

0.05759 

0.02260 

0.9111 

0.9368 

0.6775 

0.6853 

0.5797 

0.5474 

0.7989 

0.5007 

0.7917 

ITP5 

0.17519 

0.20176 

-0.07108 

-0.06914 

-0.06529 

-0.06105 

-0.02566 

-0.00902 

0.27160 

0.0391 

0.0172 

0.4057 

0.4186 

0.4451 

0.4753 

0 . 7643 

0.9161 

0.0012 

OTP5 

0.19556 

0.22374 

-0.06231 

-0.05935 

-0.07233 

-0.06891 

-0.01918 

-0.00293 

0.28086 

0.0210 

0.0081 

0.4662 

0.4877 

0.3975 

0.4202 

0.8227 

0.9726 

0.0008 

ITP7 

-0.04254 

-0.05096 

-0.08270 

-0.08395 

-0.07596 

-0.07917 

-0.02986 

0.0021 1 

-0.02986 

0.6190 

0.5513 

0.3331 

0.3258 

0.3741 

0.3542 

0.7271 

0.9804 

0.7271 

0TP7 

-0.03595 

-0.04261 

-0.07492 

-0.07575 

-0.08974 

-0.09378 

-0.01497 

0.01774 

-0.01119 

0 . 6744 

0.6185 

0.3807 

0.3755 

0.2934 

0.2722 

0.8611 

0.8358 

0.8960 

ITP8 

-0.02080 

-0.02312 

-0.04044 

-0.03842 

-0.03714 

-0.04231 

-0.01460 

0.00653 

-0.01460 

0.8080 

0.7870 

0.6365 

0.6534 

0.6642 

0.6209 

0.8646 

0.9392 

0.8646 

0TP8 

-0.02377 

-0.02528 

-0.01903 

-0.01660 

-0.05956 

-0.06558 

0.00840 

0.02994 

0.01113 

0.7812 

0.7677 

0.8241 

0.8462 

0.4861 

0.4431 

0.9218 

0.7264 

0.8966 

ITP10 

0.35622 

0.38893 

0.57700 

0.59381 

-0.10691 

-0.11972 

-0.04202 

-0.06187 

0.15268 

<.0001 

<.0001 

<.0001 

<.0001 

0.2103 

0.1604 

0.6233 

0.4693 

0.0727 

0TP10 

0.35459 

0.38835 

0.59005 

0.60711 

-0.09804 

-0.11073 

-0.04512 

-0.06502 

0.15062 

<.0001 

<.0001 

<.0001 

<.0001 

0.2509 

0.1944 

0.5979 

0.4470 

0.0767 

ITP1 1 

0.11063 

0.12989 

0.38401 

0.39031 

-0.11274 

-0.11478 

-0.10530 

-0.09846 

0.01667 

0.1948 

0.1275 

<.0001 

<.0001 

0.1864 

0.1785 

0.2173 

0.2488 

0.8456 
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Observed  to  Trained  NNS  Output  for  Exercises  1,2,3  8 

07:43  Wednesday,  January  15,  2003 

The  CORR  Procedure 

Pearson  Correlation  Coefficients,  N  =  139 
Prob  >  | r |  under  HO:  Rho=0 


OTS22 

ITS23 

0TS23 

ITA2 

OTA2 

ITA3 

OTA3 

ITA7 

0TA7 

ITP2 

0.01021 

-0.03283 

-0.03579 

-0.00725 

-0.01618 

-0.02617 

-0.02933 

-0.01808 

-0.0041 1 

0.9051 

0.7012 

0.6757 

0.9325 

0.8500 

0.7598 

0.7318 

0.8327 

0.9617 

OTP2 

0.06024 

0.01972 

0.03119 

0.05090 

0.04540 

0.33787 

0.34112 

0.09990 

0.08701 

0.4812 

0.8177 

0.7155 

0.5518 

0.5957 

<.0001 

<.0001 

0.2420 

0.3084 

ITP3 

0.00420 

-0.08192 

-0.07738 

-0.01808 

-0.04393 

-0.06529 

-0.07082 

-0.0451 1 

-0.04699 

0.9609 

0.3377 

0.3653 

0.8327 

0.6076 

0.4451 

0.4074 

0.5980 

0.5828 

OTP3 

0.00266 

-0.08618 

-0.08184 

-0.02195 

-0.04917 

-0.07331 

-0.07931 

-0.04855 

-0.05102 

0.9752 

0.3131 

0.3382 

0.7976 

0.5654 

0.3911 

0.3533 

0.5703 

0.5508 

ITP4 

-0.00297 

-0.03283 

-0.03606 

-0.00725 

-0.01725 

-0.02617 

-0.02395 

-0.01808 

-0.01422 

0.9724 

0.7012 

0.6734 

0.9325 

0 . 8403 

0.7598 

0.7796 

0.8327 

0.8680 

0TP4 

0.03463 

-0.04891 

-0.05231 

-0.01463 

-0.02954 

-0.03342 

-0.02824 

-0.00780 

-0.00181 

0.6857 

0.5675 

0.5408 

0 . 8643 

0.7300 

0.6961 

0.7414 

0.9274 

0.9832 

ITP5 

0.31647 

-0.08192 

-0.07174 

-0.01808 

-0.04571 

-0.06529 

-0.05793 

-0.0451 1 

-0.04115 

0.0001 

0.3377 

0.4013 

0.8327 

0.5932 

0.4451 

0.4982 

0.5980 

0.6306 

OTP5 

0.32699 

-0.07091 

-0.06086 

-0.02147 

-0.05248 

-0.05000 

-0.04232 

-0.04640 

-0.04266 

<.0001 

0.4068 

0.4766 

0.8019 

0.5395 

0.5589 

0.6208 

0.5876 

0.6181 

ITP7 

0.01192 

-0.09531 

-0.08636 

-0.02104 

-0.01639 

0.58396 

0.59002 

-0.05249 

-0.04319 

0.8893 

0.2644 

0.3121 

0.8058 

0.8481 

<.0001 

<.0001 

0.5394 

0.6136 

0TP7 

0.03107 

-0.08008 

-0.07178 

-0.02387 

-0.02068 

0.59610 

0.60279 

-0.04373 

-0.03458 

0.7166 

0.3487 

0.4011 

0.7803 

0.8091 

<.0001 

<.0001 

0.6092 

0.6861 

ITP8 

0.01669 

-0.04660 

-0.04100 

-0.01029 

-0.02146 

-0.03714 

-0.02682 

-0.02566 

-0.01301 

0.8454 

0.5859 

0.6318 

0 . 9043 

0.8020 

0.6642 

0.7540 

0 . 7643 

0.8792 

0TP8 

0.04193 

-0.06274 

-0.05763 

-0.01675 

-0.02881 

-0.04060 

-0.03138 

-0.01048 

0.00141 

0.6241 

0.4631 

0.5004 

0.8448 

0.7364 

0.6351 

0.7139 

0.9026 

0.9868 

ITP10 

0.16582 

-0.13415 

-0.14067 

-0.02961 

-0.05304 

-0.10691 

-0.11252 

-0.07387 

-0.09197 

0.0511 

0.1154 

0.0986 

0.7294 

0.5352 

0.2103 

0.1872 

0.3874 

0.2815 

0TP10 

0.16595 

-0.13849 

-0.14485 

-0.03151 

-0.05410 

-0.11148 

-0.11676 

-0.07932 

-0.09796 

0.0509 

0.1040 

0.0889 

0.7127 

0.5271 

0.1914 

0.1711 

0.3533 

0.2513 

ITP11 

0.02372 

0.18300 

0.17076 

-0.07419 

-0.11160 

0.04242 

0.04471 

-0.18510 

-0.19289 

0.7817 

0.0311 

0 . 0444 

0.3854 

0.1909 

0.6201 

0.6012 

0.0291 

0.0229 
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Observed  to  Trained  NNS  Output  for  Exercises  1,2,3  9 

07:43  Wednesday,  January  15,  2003 

The  CORR  Procedure 

Pearson  Correlation  Coefficients,  N  =  139 
Prob  >  | r |  under  HO:  Rho=0 


ITA10 

0TA10 

ITA1 1 

0TA1 1 

ITA12 

OTA12 

ITA18 

0TA18 

ITA19 

ITP2 

-0.01029 

-0.01502 

-0.01960 

-0.02153 

-0.01264 

0.00501 

-0.01960 

-0.02194 

-0.01264 

0 . 9043 

0.8607 

0.8188 

0.8014 

0.8826 

0.9533 

0.8188 

0.7976 

0.8826 

OTP2 

0.16474 

0.19269 

0.09092 

0.11518 

0.43531 

0.45616 

-0.06377 

-0.05739 

-0.03572 

0.0526 

0.0231 

0.2871 

0.1770 

<.0001 

<.0001 

0.4558 

0.5022 

0.6764 

ITP3 

-0.02566 

-0.02056 

-0.04891 

-0.05099 

0.69926 

0.71382 

-0.04891 

-0.04246 

-0.03155 

0.7643 

0.8102 

0.5675 

0.5511 

<.0001 

<.0001 

0.5675 

0.6197 

0.7124 

OTP3 

-0.03115 

-0.02677 

-0.05513 

-0.05682 

0.70014 

0.71661 

-0.04596 

-0.03912 

-0.03293 

0.7158 

0 . 7544 

0.5192 

0.5064 

<.0001 

<.0001 

0.5911 

0.6475 

0.7004 

ITP4 

-0.01029 

0.00273 

-0.01960 

-0.00552 

-0.01264 

-0.01644 

-0.01960 

-0.02318 

-0.01264 

0 . 9043 

0.9746 

0.8188 

0.9486 

0.8826 

0.8477 

0.8188 

0.7865 

0.8826 

0TP4 

0.01537 

0.03010 

0.00197 

0.01766 

-0.01697 

-0.01798 

-0.03520 

-0.03919 

-0.02421 

0.8574 

0.7250 

0.9816 

0.8365 

0.8428 

0.8336 

0.6808 

0.6469 

0.7773 

ITP5 

-0.02566 

-0.0021 1 

0.11296 

0.11977 

0.69926 

0.71229 

-0.04891 

-0.04536 

-0.03155 

0 . 7643 

0.9804 

0.1855 

0.1602 

<.0001 

<.0001 

0.5675 

0.5959 

0.7124 

OTP5 

-0.02420 

0.00003 

0.12478 

0.13312 

0.70071 

0.71754 

-0.04475 

-0.04129 

-0.03720 

0.7774 

0.9997 

0.1433 

0.1182 

<.0001 

<.0001 

0.6009 

0.6294 

0.6638 

ITP7 

-0.02986 

-0.01381 

-0.05691 

-0.05550 

-0.03670 

-0.03950 

-0.05691 

-0.06162 

-0.03670 

0.7271 

0.8718 

0.5058 

0.5164 

0.6680 

0 . 6443 

0.5058 

0.4711 

0.6680 

0TP7 

-0.02740 

-0.01093 

-0.06487 

-0.06281 

-0.04297 

-0.04421 

-0.04530 

-0.05025 

-0.03633 

0.7489 

0.8984 

0.4481 

0.4626 

0.6154 

0.6053 

0.5964 

0.5569 

0.6711 

ITP8 

-0.01460 

-0.02079 

-0.02782 

-0.03376 

-0.01795 

-0.01507 

-0.02782 

-0.02853 

-0.01795 

0.8646 

0.8081 

0.7451 

0.6931 

0.8339 

0.8602 

0.7451 

0.7389 

0.8339 

0TP8 

-0.02378 

-0.03040 

-0.04413 

-0.05067 

-0.03012 

-0.02484 

-0.04116 

-0.04256 

-0.02699 

0.7812 

0.7224 

0.6060 

0.5536 

0.7249 

0.7716 

0.6305 

0.6188 

0.7525 

ITP10 

-0.04202 

-0.05128 

-0.08009 

-0.06504 

-0.05166 

-0.01141 

-0.08009 

-0.08787 

-0.05166 

0.6233 

0.5488 

0.3486 

0.4469 

0.5459 

0.8940 

0.3486 

0.3037 

0.5459 

0TP10 

-0.03750 

-0.04649 

-0.07431 

-0.05941 

-0.03591 

0.00438 

-0.07835 

-0.08608 

-0.04922 

0.6612 

0.5868 

0.3846 

0.4873 

0.6747 

0.9592 

0.3592 

0.3137 

0.5650 

ITP1 1 

-0.10530 

-0.12774 

-0.20069 

-0.20528 

-0.12944 

-0.09835 

0.19782 

0.19185 

-0.02948 

0.2173 

0.1340 

0.0178 

0.0153 

0.1289 

0.2494 

0.0196 

0.0237 

0.7304 
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Observed  to  Trained  NNS  Output  for  Exercises  1,2,3  10 

07:43  Wednesday,  January  15,  2003 

The  CORR  Procedure 

Pearson  Correlation  Coefficients,  N  =  139 
Prob  >  | r |  under  HO:  Rho=0 


0TA19 

ITA21 

0TA21 

ITA23 

OTA23 

IM1 

0M1 

IM2 

0M2 

ITP2 

-0.01536 

-0.03691 

-0.03925 

0.17338 

0.16871 

-0.04331 

-0.04246 

0.03204 

0.01129 

0.8575 

0.6662 

0.6464 

0.0412 

0.0471 

0.6127 

0.6197 

0.7081 

0.8950 

OTP2 

-0.04824 

-0.23864 

-0.24296 

0.21602 

0.23115 

0.13301 

0.13620 

-0.43835 

-0.43662 

0.5728 

0.0047 

0.0040 

0.0106 

0.0062 

0.1185 

0.1099 

<.0001 

<•0001 

ITP3 

-0.04665 

-0.09210 

-0.09673 

0.16415 

0.16721 

-0.10805 

-0.10083 

0.14789 

0.14001 

0.5856 

0.2809 

0.2573 

0.0535 

0.0491 

0.2055 

0.2376 

0.0823 

0.1002 

OTP3 

-0.04836 

-0.10682 

-0.11145 

0.18041 

0.18360 

-0.11261 

-0.10517 

0.14638 

0.13864 

0.5718 

0.2107 

0.1915 

0.0336 

0.0305 

0.1869 

0.2179 

0.0855 

0.1036 

ITP4 

-0.01911 

-0.03691 

-0.03949 

0.17338 

0.17117 

-0.04331 

-0.04406 

0.15459 

0.13656 

0.8233 

0.6662 

0 . 6444 

0.0412 

0.0439 

0.6127 

0.6065 

0.0692 

0.1089 

0TP4 

-0.03077 

-0.03234 

-0.03571 

0.23133 

0.22953 

-0.02963 

-0.03090 

0.14522 

0.13066 

0.7191 

0.7055 

0.6764 

0.0061 

0.0066 

0.7292 

0.7180 

0.0881 

0.1252 

ITP5 

-0.04223 

-0.09210 

-0.09743 

0.07467 

0.07851 

-0.10805 

-0.09773 

0.13090 

0.14534 

0.6216 

0.2809 

0.2538 

0.3823 

0.3583 

0.2055 

0.2524 

0.1246 

0.0878 

OTP5 

-0.05006 

-0.09994 

-0.10599 

0.09853 

0.10295 

-0.09185 

-0.08191 

0.11966 

0.13436 

0.5584 

0.2418 

0.2143 

0.2485 

0.2278 

0.2822 

0.3378 

0.1606 

0.1148 

ITP7 

-0.03146 

-0.10716 

-0.09748 

0.03483 

0.03542 

0.04231 

0.04612 

-0.26279 

-0.26696 

0.7131 

0.2093 

0.2536 

0.6840 

0.6789 

0.6209 

0.5898 

0.0018 

0.0015 

0TP7 

-0.03120 

-0.09489 

-0.08540 

0.05088 

0.05198 

0.04942 

0.05378 

-0.27007 

-0.27422 

0.7154 

0.2665 

0.3175 

0.5519 

0.5434 

0.5634 

0.5295 

0.0013 

0.0011 

ITP8 

-0.01915 

-0.05239 

-0.04263 

0.24608 

0.24676 

-0.06147 

-0.06367 

-0.12848 

-0.12142 

0.8230 

0.5402 

0.6183 

0.0035 

0.0034 

0.4722 

0.4565 

0.1317 

0.1545 

0TP8 

-0.02973 

-0.04369 

-0.03306 

0.27618 

0.27674 

-0.06920 

-0.07130 

-0.11983 

-0.11394 

0.7283 

0.6095 

0.6992 

0.0010 

0.0010 

0.4183 

0.4042 

0.1600 

0.1817 

ITP10 

-0.07604 

-0.15082 

-0.15086 

0.70837 

0.71743 

-0.17694 

-0.18147 

-0.14730 

-0.15091 

0.3736 

0.0764 

0.0763 

<.0001 

<.0001 

0.0372 

0.0325 

0.0836 

0.0762 

0TP10 

-0.07362 

-0.15245 

-0.15254 

0.71315 

0.72296 

-0.18469 

-0.18903 

-0.15906 

-0.16208 

0.3890 

0.0732 

0.0730 

<.0001 

<.0001 

0.0295 

0.0258 

0.0615 

0.0566 

ITP11 

-0.04457 

0.49757 

0.49300 

0.45325 

0.46531 

0.12287 

0.10598 

-0.08330 

-0.07137 

0.6024 

<.0001 

<.0001 

<.0001 

<.0001 

0.1496 

0.2143 

0.3296 

0.4037 
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Observed  to  Trained  NNS  Output  for  Exercises  1,2,3 

07:43  Wednesday,  January  15, 

The  CORR  Procedure 

Pearson  Correlation  Coefficients,  N  =  139 
Prob  >  | r |  under  HO:  Rho=0 


I  M3 

0M3 

IM4 

0M4 

IM5 

0M5 

ITP2 

-0.03222 

-0.02047 

-0.01029 

-0.02747 

-0.02795 

-0.02869 

0.7065 

0.8110 

0.9043 

0.7482 

0.7440 

0.7374 

OTP2 

0.30661 

0.32194 

0.16474 

0.19645 

0.50923 

0.52162 

0.0002 

0.0001 

0.0526 

0.0205 

<.0001 

<.0001 

ITP3 

-0.08039 

-0.06549 

-0.02566 

-0.02400 

-0.06974 

-0.07882 

0.3469 

0.4437 

0.7643 

0.7792 

0.4146 

0.3563 

OTP3 

-0.06992 

-0.05523 

-0.03115 

-0.02958 

-0.06982 

-0.07931 

0.4134 

0.5184 

0.7158 

0.7296 

0.4141 

0.3534 

ITP4 

-0.03222 

-0.01969 

-0.01029 

0.01617 

-0.02795 

-0.01256 

0.7065 

0.8181 

0 . 9043 

0.8502 

0 . 7440 

0.8834 

0TP4 

-0.00039 

0.01392 

0.01537 

0.04454 

-0.01982 

-0.00119 

0.9964 

0.8708 

0.8574 

0.6026 

0.8169 

0.9889 

ITP5 

0.05491 

0.06009 

-0.02566 

0.05601 

-0.06974 

-0.06781 

0.5209 

0.4823 

0 . 7643 

0.5125 

0.4146 

0.4277 

OTP5 

0.07472 

0.08105 

-0.02420 

0.06085 

-0.07042 

-0.06745 

0.3820 

0.3429 

0.7774 

0.4768 

0.4101 

0.4302 

ITP7 

-0.09353 

-0.08854 

-0.02986 

-0.03800 

0.63814 

0.64104 

0.2735 

0.3000 

0.7271 

0.6570 

<.0001 

<.0001 

0TP7 

-0.07470 

-0.07010 

-0.02740 

-0.03786 

0.64759 

0.65306 

0.3821 

0.4122 

0.7489 

0.6581 

<.0001 

<.0001 

ITP8 

-0.04573 

-0.02435 

-0.01460 

-0.04180 

0.35421 

0.33721 

0.5930 

0.7760 

0.8646 

0.6251 

<.0001 

<.0001 

0TP8 

-0.02145 

-0.00084 

-0.02378 

-0.05718 

0.35747 

0.34426 

0.8021 

0.9921 

0.7812 

0.5038 

<■0001 

<.0001 

ITP10 

0.73371 

0.73487 

-0.04202 

-0.04092 

0.40412 

0.42151 

<.0001 

<.0001 

0.6233 

0.6325 

<.0001 

<.0001 

0TP10 

0.73920 

0.74111 

-0.03750 

-0.03707 

0.41659 

0.43320 

<.0001 

<.0001 

0.6612 

0.6649 

<.0001 

<.0001 

ITP1 1 

0.21223 

0.21630 

-0.10530 

-0.16277 

0.13324 

0.14285 

0.0121 

0.0105 

0.2173 

0.0556 

0.1179 

0.0934 

1 1 
2003 
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Observed  to  Trained  NNS  Output  for  Exercises  1,2,3  12 

07:43  Wednesday,  January  15,  2003 

The  CORR  Procedure 

Pearson  Correlation  Coefficients,  N  =  139 
Prob  >  | r |  under  HO:  Rho=0 


ITP2 

OTP2 

ITP3 

OTP3 

ITP4 

0TP4 

ITP5 

OTP5 

ITP7 

OTP11 

0.09203 

0.00611 

0.02823 

0.03577 

-0.06910 

-0.04418 

-0.17942 

-0.15918 

-0.08769 

0.2812 

0.9431 

0.7414 

0.6759 

0.4189 

0.6056 

0.0346 

0.0613 

0.3047 

ITP12 

-0.02617 

-0.0431 1 

-0.06529 

-0.04170 

-0.02617 

-0.03614 

-0.06529 

-0.04973 

-0.07596 

0.7598 

0.6143 

0.4451 

0.6260 

0.7598 

0.6727 

0.4451 

0.5610 

0.3741 

OTP12 

-0.02709 

-0.03612 

-0.05126 

-0.02768 

-0.02994 

-0.04023 

-0.06193 

-0.04601 

-0.07175 

0.7516 

0.6729 

0.5490 

0.7463 

0.7264 

0.6382 

0.4689 

0.5907 

0.4013 

ITP15 

-0.02734 

0.38661 

-0.06822 

-0.06267 

-0.02734 

0.00622 

-0.06822 

-0.05664 

-0.07938 

0.7493 

<.0001 

0.4249 

0.4636 

0.7493 

0.9421 

0.4249 

0.5078 

0.3530 

OTP15 

-0.02951 

0.4041 1 

-0.05921 

-0.05357 

-0.01764 

0.01526 

-0.05389 

-0.04166 

-0.07485 

0.7302 

<.0001 

0.4887 

0.5311 

0.8367 

0.8585 

0.5287 

0.6263 

0.3812 

ITP22 

-0.01960 

0.09092 

-0.04891 

-0.05649 

-0.01960 

-0.03564 

-0.04891 

-0.05756 

-0.05691 

0.8188 

0.2871 

0.5675 

0.5089 

0.8188 

0.6770 

0.5675 

0.5009 

0.5058 

OTP22 

-0.01952 

0.09377 

-0.05189 

-0.05975 

-0.02275 

-0.03942 

-0.05508 

-0.06406 

-0.05372 

0.8196 

0.2722 

0.5441 

0.4847 

0.7904 

0.6449 

0.5196 

0.4538 

0.5299 

ITP23 

-0.02961 

-0.00757 

-0.07387 

-0.07005 

-0.02961 

-0.04282 

-0.07387 

-0.07387 

-0.08595 

0.7294 

0.9295 

0.3874 

0.4126 

0.7294 

0.6167 

0.3874 

0.3875 

0.3144 

OTP23 

-0.01763 

0.00247 

-0.05381 

-0.04972 

-0.03073 

-0.04346 

-0.06069 

-0.06031 

-0.08260 

0.8368 

0.9770 

0.5292 

0.5611 

0.7195 

0.6115 

0.4779 

0.4806 

0.3337 

ITS1 

-0.00725 

0.16261 

-0.01808 

-0.01492 

-0.00725 

-0.00558 

-0.01808 

-0.02102 

-0.02104 

0.9325 

0.0558 

0.8327 

0.8616 

0.9325 

0.9481 

0.8327 

0.8059 

0.8058 

OTS1 

-0.01111 

0.17476 

0.00696 

0.01022 

-0.00570 

-0.00285 

-0.01241 

-0.01454 

-0.03422 

0.8967 

0.0396 

0.9352 

0.9050 

0.9469 

0.9735 

0.8847 

0.8651 

0.6892 

ITS2 

-0.01808 

0.40186 

-0.0451 1 

-0.05422 

-0.01808 

-0.03321 

-0.0451 1 

-0.04437 

0.85949 

0.8327 

<.0001 

0.5980 

0.5261 

0.8327 

0.6980 

0.5980 

0.6040 

<.0001 

OTS2 

-0.01825 

0.40547 

-0.04968 

-0.05921 

-0.01347 

-0.02732 

-0.04119 

-0.04024 

0.85715 

0.8312 

<.0001 

0.5614 

0.4887 

0.8749 

0.7496 

0.6302 

0.6381 

<.0001 

ITS5 

-0.01808 

-0.07043 

-0.0451 1 

-0.05038 

0.40078 

0.45948 

-0.0451 1 

-0.03262 

-0.05249 

0.8327 

0.4100 

0.5980 

0.5559 

<.0001 

<.0001 

0.5980 

0.7031 

0.5394 

OTS5 

-0.02174 

-0.07346 

-0.05259 

-0.05832 

0.39748 

0.46063 

-0.03280 

-0.02106 

-0.05591 

0.7995 

0.3901 

0.5386 

0.4953 

<.0001 

<.0001 

0.7015 

0.8056 

0.5133 
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Observed  to  Trained  NNS  Output  for  Exercises  1,2,3  13 

07:43  Wednesday,  January  15,  2003 

The  CORR  Procedure 

Pearson  Correlation  Coefficients,  N  =  139 
Prob  >  | r |  under  HO:  Rho=0 


0TP7 

ITP8 

0TP8 

ITP10 

0TP10 

ITP1 1 

OTP1 1 

ITP12 

0TP12 

OTP1 1 

-0.06008 

0.14321 

0.16978 

0.40622 

0.41139 

0.99938 

1 .00000 

0.35551 

0.37533 

0.4824 

0.0926 

0.0457 

<.0001 

<.0001 

<.0001 

<.0001 

<.0001 

ITP12 

-0.06191 

-0.03714 

-0.03852 

0.22333 

0.22656 

0.35272 

0.35551 

1 .00000 

0.99855 

0.4690 

0.6642 

0.6526 

0.0082 

0.0073 

<.0001 

<.0001 

<.0001 

OTP12 

-0.05742 

-0.02789 

-0.02867 

0.22699 

0.23082 

0.37205 

0.37533 

0.99855 

1 .00000 

0.5019 

0.7445 

0.7376 

0.0072 

0.0063 

<.0001 

<.0001 

<.0001 

ITP15 

-0.06996 

-0.03881 

-0.02908 

0.60499 

0.61657 

0.16903 

0.16845 

-0.09874 

-0.09947 

0.4131 

0.6501 

0.7340 

<.0001 

<.0001 

0.0467 

0.0474 

0.2475 

0.2440 

OTP15 

-0.06522 

-0.03843 

-0.02823 

0.61974 

0.63150 

0.16322 

0.16282 

-0.09585 

-0.09679 

0.4456 

0.6533 

0.7415 

<.0001 

<.0001 

0.0549 

0.0555 

0.2617 

0.2570 

ITP22 

-0.05634 

-0.02782 

-0.03433 

-0.08009 

-0.08578 

-0.20069 

-0.20459 

-0.07079 

-0.07831 

0.5101 

0.7451 

0.6883 

0.3486 

0.3154 

0.0178 

0.0157 

0.4076 

0.3595 

OTP22 

-0.05338 

-0.03114 

-0.03780 

-0.08524 

-0.09107 

-0.22076 

-0.22521 

-0.07911 

-0.08744 

0.5326 

0.7159 

0.6587 

0.3184 

0.2863 

0.0090 

0.0077 

0.3546 

0.3060 

ITP23 

-0.09649 

-0.04202 

-0.05776 

-0.12097 

-0.11388 

-0.30311 

-0.30619 

-0.10691 

-0.11026 

0.2585 

0.6233 

0.4994 

0.1560 

0.1819 

0.0003 

0.0002 

0.2103 

0.1963 

OTP23 

-0.09339 

-0.04261 

-0.05850 

-0.12286 

-0.11547 

-0.30084 

-0.30417 

-0.10506 

-0.10812 

0.2742 

0.6184 

0.4939 

0.1496 

0.1759 

0.0003 

0.0003 

0.2184 

0.2052 

ITS1 

-0.01892 

-0.01029 

0.00630 

-0.02961 

-0.03179 

-0.07419 

-0.07412 

-0.02617 

-0.02963 

0.8250 

0 . 9043 

0.9413 

0.7294 

0.7103 

0.3854 

0.3859 

0.7598 

0.7292 

OTS1 

-0.03176 

-0.01270 

0.00443 

-0.04599 

-0.04796 

-0.08678 

-0.08750 

-0.03784 

-0.04139 

0.7106 

0.8821 

0.9587 

0.5908 

0.5750 

0.3097 

0.3057 

0.6583 

0.6285 

ITS2 

0.86513 

-0.02566 

-0.02901 

-0.07387 

-0.07544 

-0.18510 

-0.18441 

-0.06529 

-0.06588 

<.0001 

0 . 7643 

0.7346 

0.3874 

0.3774 

0.0291 

0.0298 

0.4451 

0.4410 

OTS2 

0.86334 

-0.02805 

-0.0321 1 

-0.08082 

-0.08227 

-0.18384 

-0.18370 

-0.06354 

-0.06456 

<.0001 

0.7430 

0.7074 

0.3443 

0.3357 

0.0303 

0.0304 

0.4574 

0.4502 

ITS5 

-0.05481 

-0.02566 

-0.04275 

-0.07387 

-0.06992 

0.02931 

0.02414 

-0.06529 

-0.07471 

0.5216 

0 . 7643 

0.6173 

0.3874 

0.4134 

0.7320 

0.7779 

0.4451 

0.3821 

OTS5 

-0.05733 

-0.03054 

-0.04763 

-0.07442 

-0.07090 

0.03400 

0.02878 

-0.07439 

-0.08443 

0.5026 

0.7211 

0.5776 

0.3840 

0.4069 

0.6911 

0.7366 

0.3841 

0.3231 
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Observed  to  Trained  NNS  Output  for  Exercises  1,2,3  14 

07:43  Wednesday,  January  15,  2003 

The  CORR  Procedure 

Pearson  Correlation  Coefficients,  N  =  139 
Prob  >  | r |  under  HO:  Rho=0 


ITP15 

0TP15 

ITP22 

0TP22 

ITP23 

0TP23 

ITS1 

OTS1 

ITS2 

OTP11 

0.16845 

0.16282 

-0.20459 

-0.22521 

-0.30619 

-0.30417 

-0.07412 

-0.08750 

-0.18441 

0.0474 

0.0555 

0.0157 

0.0077 

0.0002 

0.0003 

0.3859 

0.3057 

0.0298 

ITP12 

-0.09874 

-0.09585 

-0.07079 

-0.07911 

-0.10691 

-0.10506 

-0.02617 

-0.03784 

-0.06529 

0.2475 

0.2617 

0.4076 

0.3546 

0.2103 

0.2184 

0.7598 

0.6583 

0.4451 

OTP12 

-0.09947 

-0.09679 

-0.07831 

-0.08744 

-0.11026 

-0.10812 

-0.02963 

-0.04139 

-0.06588 

0.2440 

0.2570 

0.3595 

0.3060 

0.1963 

0.2052 

0.7292 

0.6285 

0.4410 

ITP15 

1  .00000 

0.99877 

-0.07397 

-0.07481 

-0.11172 

-0.11696 

-0.02734 

-0.03187 

-0.06822 

<.0001 

0.3868 

0.3814 

0.1904 

0.1703 

0.7493 

0.7096 

0.4249 

OTP15 

0.99877 

1 .00000 

-0.06584 

-0.06661 

-0.10607 

-0.11090 

-0.02890 

-0.03355 

-0.06326 

<■0001 

0.4413 

0.4359 

0.2139 

0.1937 

0.7355 

0.6950 

0.4594 

ITP22 

-0.07397 

-0.06584 

1 .00000 

0.99806 

-0.08009 

-0.06794 

0.36966 

0.38979 

-0.04891 

0.3868 

0.4413 

<.0001 

0.3486 

0.4268 

<.0001 

<.0001 

0.5675 

OTP22 

-0.07481 

-0.06661 

0.99806 

1 .00000 

-0.05739 

-0.04604 

0.35832 

0.38215 

-0.04332 

0.3814 

0.4359 

<.0001 

0.5022 

0.5904 

<.0001 

<.0001 

0.6126 

ITP23 

-0.11172 

-0.10607 

-0.08009 

-0.05739 

1 .00000 

0.99886 

-0.02961 

-0.00144 

-0.07387 

0.1904 

0.2139 

0.3486 

0.5022 

<.0001 

0.7294 

0.9866 

0.3874 

OTP23 

-0.11696 

-0.11090 

-0.06794 

-0.04604 

0.99886 

1 .00000 

-0.01763 

0.00987 

-0.06969 

0.1703 

0.1937 

0.4268 

0.5904 

<.0001 

0.8368 

0.9082 

0.4149 

ITS1 

-0.02734 

-0.02890 

0.36966 

0.35832 

-0.02961 

-0.01763 

1 .00000 

0.99189 

-0.01808 

0.7493 

0.7355 

<.0001 

<.0001 

0.7294 

0.8368 

<.0001 

0.8327 

OTS1 

-0.03187 

-0.03355 

0.38979 

0.38215 

-0.00144 

0.00987 

0.99189 

1 .00000 

-0.03178 

0.7096 

0.6950 

<.0001 

<.0001 

0.9866 

0.9082 

<.0001 

0.7103 

ITS2 

-0.06822 

-0.06326 

-0.04891 

-0.04332 

-0.07387 

-0.06969 

-0.01808 

-0.03178 

1.00000 

0.4249 

0.4594 

0.5675 

0.6126 

0.3874 

0.4149 

0.8327 

0.7103 

OTS2 

-0.07112 

-0.06589 

-0.03309 

-0.02696 

-0.05965 

-0.05495 

-0.02085 

-0.03398 

0.99872 

0.4054 

0.4409 

0.6990 

0.7527 

0.4854 

0.5205 

0.8075 

0.6913 

<.0001 

ITS5 

0.17493 

0.17275 

-0.04891 

-0.04785 

-0.07387 

-0.07088 

-0.01808 

-0.02007 

-0.0451 1 

0.0394 

0.0420 

0.5675 

0.5759 

0.3874 

0.4070 

0.8327 

0.8146 

0.5980 

OTS5 

0.18014 

0.17797 

-0.04672 

-0.04612 

-0.08150 

-0.07864 

-0.01248 

-0.01383 

-0.04618 

0.0338 

0.0361 

0.5850 

0.5898 

0.3402 

0.3575 

0.8840 

0.8717 

0.5893 
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Observed  to  Trained  NNS  Output  for  Exercises  1,2,3  15 

07:43  Wednesday,  January  15,  2003 

The  CORR  Procedure 

Pearson  Correlation  Coefficients,  N  =  139 
Prob  >  | r |  under  HO:  Rho=0 


OTS2 

ITS5 

OTS5 

ITS8 

0TS8 

ITS1 1 

0TS11 

ITS12 

0TS12 

OTP1 1 

-0.18370 

0.02414 

0.02878 

-0.06928 

0.18595 

-0.16656 

-0.16791 

0.43666 

0.44566 

0.0304 

0.7779 

0.7366 

0.4177 

0.0284 

0.0500 

0.0482 

<.0001 

<.0001 

ITP12 

-0.06354 

-0.06529 

-0.07439 

-0.02617 

0.49150 

-0.05938 

-0.04233 

-0.09052 

-0.08622 

0.4574 

0.4451 

0.3841 

0.7598 

<.0001 

0.4875 

0.6208 

0.2893 

0.3129 

OTP12 

-0.06456 

-0.07471 

-0.08443 

-0.02963 

0.50524 

-0.05529 

-0.03800 

-0.07653 

-0.07197 

0.4502 

0.3821 

0.3231 

0.7292 

<.0001 

0.5180 

0.6570 

0.3706 

0.3998 

ITP15 

-0.07112 

0.17493 

0.18014 

-0.02734 

-0.06429 

0.20332 

0.22105 

-0.16133 

-0.16225 

0.4054 

0.0394 

0.0338 

0.7493 

0.4521 

0.0164 

0.0089 

0.0578 

0.0564 

OTP15 

-0.06589 

0.17275 

0.17797 

-0.02951 

-0.06000 

0.21602 

0.23475 

-0.16335 

-0.16445 

0.4409 

0.0420 

0.0361 

0.7302 

0.4829 

0.0106 

0.0054 

0.0547 

0.0531 

ITP22 

-0.03309 

-0.04891 

-0.04672 

-0.01960 

-0.14070 

-0.04448 

-0.04876 

-0.11566 

-0.11692 

0.6990 

0.5675 

0.5850 

0.8188 

0.0985 

0.6031 

0.5687 

0.1751 

0.1705 

OTP22 

-0.02696 

-0.04785 

-0.04612 

-0.02275 

-0.14998 

-0.04135 

-0.0461 1 

-0.12715 

-0.12850 

0.7527 

0.5759 

0.5898 

0.7904 

0.0780 

0.6289 

0.5899 

0.1358 

0.1317 

ITP23 

-0.05965 

-0.07387 

-0.08150 

-0.02961 

-0.12718 

-0.06718 

-0.04937 

0.05656 

0.05413 

0.4854 

0.3874 

0.3402 

0.7294 

0.1357 

0.4320 

0.5638 

0.5084 

0.5268 

OTP23 

-0.05495 

-0.07088 

-0.07864 

-0.02560 

-0.11891 

-0.05836 

-0.04030 

0.05039 

0.04788 

0.5205 

0.4070 

0.3575 

0.7648 

0.1633 

0.4950 

0.6376 

0.5558 

0.5757 

ITS1 

-0.02085 

-0.01808 

-0.01248 

-0.00725 

-0.02885 

-0.01644 

-0.00580 

-0.04275 

-0.04274 

0.8075 

0.8327 

0.8840 

0.9325 

0.7360 

0.8476 

0.9460 

0.6173 

0.6173 

OTS1 

-0.03398 

-0.02007 

-0.01383 

-0.00895 

-0.03029 

0.00330 

0.01448 

-0.04251 

-0.04193 

0.6913 

0.8146 

0.8717 

0.9168 

0.7234 

0.9693 

0.8656 

0.6193 

0.6241 

ITS2 

0.99872 

-0.0451 1 

-0.04618 

-0.01808 

-0.11457 

-0.04103 

-0.05307 

-0.10668 

-0.11154 

<.0001 

0.5980 

0.5893 

0.8327 

0.1793 

0.6315 

0.5349 

0.2113 

0.1911 

OTS2 

1 .00000 

-0.03867 

-0.03935 

-0.01130 

-0.11766 

-0.04416 

-0.05587 

-0.10702 

-0.11182 

0.6513 

0.6456 

0.8950 

0.1677 

0.6057 

0.5136 

0.2099 

0.1900 

ITS5 

-0.03867 

1.00000 

0.99818 

-0.01808 

-0.12876 

-0.04103 

-0.03880 

-0.10668 

-0.11200 

0.6513 

<.0001 

0.8327 

0.1309 

0.6315 

0.6502 

0.2113 

0.1893 

OTS5 

-0.03935 

0.99818 

1 .00000 

-0.00014 

-0.14173 

-0.03773 

-0.03576 

-0.10077 

-0.10587 

0.6456 

<.0001 

0.9987 

0.0960 

0.6592 

0.6760 

0.2379 

0.2148 
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Observed  to  Trained  NNS  Output  for  Exercises  1,2,3  16 

07:43  Wednesday,  January  15,  2003 

The  CORR  Procedure 

Pearson  Correlation  Coefficients,  N  =  139 
Prob  >  | r |  under  HO:  Rho=0 


ITS15 

0TS15 

ITS18 

0TS18 

ITS19 

0TS19 

ITS20 

0TS20 

ITS22 

OTP11 

0.11270 

0.13275 

0.38154 

0.38810 

-0.11958 

-0.12173 

-0.10634 

-0.09934 

0.01932 

0.1865 

0 . 1 1 93 

<.0001 

<.0001 

0.1609 

0.1534 

0.2128 

0.2446 

0.8214 

ITP12 

0.40676 

0.43082 

0.06736 

0.08780 

-0.09449 

-0.10163 

-0.03714 

-0.05349 

-0.03714 

<.0001 

<.0001 

0.4308 

0.3041 

0.2685 

0.2339 

0.6642 

0.5317 

0.6642 

OTP12 

0.39927 

0.42460 

0.07312 

0.09375 

-0.09602 

-0.10303 

-0.04250 

-0.05865 

-0.03193 

<.0001 

<.0001 

0.3923 

0.2723 

0.2608 

0.2274 

0.6194 

0.4928 

0.7090 

ITP15 

-0.05529 

-0.02238 

0.63140 

0.63125 

-0.09874 

-0.11064 

-0.03881 

-0.05822 

-0.03881 

0.5180 

0.7937 

<.0001 

<.0001 

0.2475 

0.1948 

0.6501 

0.4960 

0.6501 

OTP15 

-0.03798 

-0.00463 

0.62909 

0.62985 

-0.09576 

-0.10782 

-0.04275 

-0.06239 

-0.03065 

0.6571 

0.9569 

<.0001 

<.0001 

0.2621 

0.2064 

0.6173 

0.4656 

0.7202 

ITP22 

-0.03964 

-0.04829 

-0.07707 

-0.08065 

-0.07079 

-0.05611 

-0.02782 

0.01076 

-0.02782 

0.6431 

0.5724 

0.3672 

0.3453 

0.4076 

0.5118 

0.7451 

0.9000 

0.7451 

OTP22 

-0.04600 

-0.05568 

-0.08104 

-0.08476 

-0.06135 

-0.04654 

-0.00533 

0.03139 

-0.02369 

0.5908 

0.5150 

0.3430 

0.3211 

0.4731 

0.5865 

0.9503 

0.7138 

0.7820 

ITP23 

-0.05987 

-0.07578 

-0.11640 

-0.10771 

0.63612 

0.63971 

0.34739 

0.36068 

-0.04202 

0.4839 

0.3753 

0.1724 

0.2069 

<.0001 

<.0001 

<.0001 

<.0001 

0.6233 

OTP23 

-0.05927 

-0.07490 

-0.10970 

-0.10096 

0.63910 

0 . 64333 

0.34023 

0.35564 

-0.03998 

0.4883 

0.3808 

0.1986 

0.2370 

<.0001 

<.0001 

<.0001 

<.0001 

0 . 6403 

ITS1 

-0.01465 

-0.01816 

-0.02849 

-0.02825 

-0.02617 

-0.02854 

-0.01029 

0.07974 

-0.01029 

0.8641 

0.8320 

0.7392 

0.7413 

0.7598 

0.7387 

0.9043 

0.3507 

0.9043 

OTS1 

-0.0241 1 

-0.02825 

-0.02300 

-0.02301 

-0.03752 

-0.03913 

0.06648 

0.15366 

-0.01270 

0.7781 

0.7413 

0.7881 

0.7880 

0.6610 

0.6474 

0.4368 

0.0709 

0.8821 

ITS2 

-0.03656 

-0.04486 

-0.07108 

-0.07369 

-0.06529 

-0.06593 

-0.02566 

-0.00141 

-0.02566 

0.6692 

0.6000 

0.4057 

0.3886 

0.4451 

0.4406 

0 . 7643 

0.9868 

0 . 7643 

OTS2 

-0.04107 

-0.04939 

-0.07308 

-0.07577 

-0.05505 

-0.05506 

-0.01604 

0.00981 

-0.01727 

0.6312 

0.5637 

0.3926 

0.3753 

0.5198 

0.5197 

0.8513 

0.9088 

0.8401 

ITS5 

-0.03656 

-0.01691 

-0.07108 

-0.07393 

-0.06529 

-0.06105 

-0.02566 

-0.00836 

-0.02566 

0.6692 

0.8433 

0.4057 

0.3871 

0.4451 

0.4753 

0 . 7643 

0.9222 

0.7643 

OTS5 

-0.03259 

-0.01315 

-0.07383 

-0.07700 

-0.07015 

-0.06640 

-0.01647 

0.00250 

-0.01647 

0.7033 

0.8779 

0.3877 

0.3676 

0.4119 

0.4374 

0 . 8474 

0.9767 

0 . 8474 
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Observed  to  Trained  NNS  Output  for  Exercises  1,2,3  17 

07:43  Wednesday,  January  15,  2003 

The  CORR  Procedure 

Pearson  Correlation  Coefficients,  N  =  139 
Prob  >  | r |  under  HO:  Rho=0 


0TS22 

ITS23 

0TS23 

ITA2 

OTA2 

ITA3 

OTA3 

ITA7 

0TA7 

OTP1 1 

0.02635 

0.17944 

0.16700 

-0.07788 

-0.11595 

0 . 04043 

0.04249 

-0.19044 

-0.19848 

0.7582 

0.0345 

0.0494 

0.3622 

0.1740 

0.6365 

0.6195 

0.0247 

0.0192 

ITP12 

-0.01724 

0.33921 

0.34278 

-0.02617 

-0.02536 

-0.09449 

-0.09097 

-0.06529 

-0.08189 

0.8404 

<.0001 

<.0001 

0.7598 

0.7669 

0.2685 

0.2868 

0.4451 

0.3379 

OTP12 

-0.01151 

0.34546 

0.34868 

-0.03026 

-0.03125 

-0.08972 

-0.08642 

-0.07497 

-0.09246 

0.8930 

<.0001 

<.0001 

0.7236 

0.7149 

0.2935 

0.3118 

0.3804 

0.2790 

ITP15 

-0.03278 

-0.12389 

-0.12604 

-0.02734 

-0.03288 

-0.09874 

-0.09112 

-0.06822 

-0.08325 

0.7016 

0.1462 

0.1393 

0.7493 

0.7008 

0.2475 

0.2861 

0.4249 

0.3299 

OTP15 

-0.02265 

-0.12468 

-0.12669 

-0.01825 

-0.02235 

-0.09814 

-0.09047 

-0.06933 

-0.08471 

0.7913 

0.1436 

0.1372 

0.8312 

0.7940 

0.2504 

0.2895 

0.4173 

0.3214 

ITP22 

-0.01371 

0.49908 

0.50055 

0.36966 

0.42595 

-0.07079 

-0.05575 

0.59858 

0.61349 

0.8727 

<.0001 

<.0001 

<.0001 

<.0001 

0.4076 

0.5145 

<.0001 

<.0001 

OTP22 

-0.00881 

0.49188 

0.49454 

0.37610 

0.43439 

-0.07230 

-0.05731 

0.61256 

0.62827 

0.9180 

<.0001 

<.0001 

<.0001 

<.0001 

0.3977 

0.5028 

<.0001 

<.0001 

ITP23 

-0.03196 

-0.13415 

-0.13329 

-0.02961 

0.01468 

-0.10691 

-0.11028 

0.15431 

0.14838 

0.7088 

0.1154 

0.1178 

0.7294 

0.8638 

0.2103 

0.1962 

0.0697 

0.0813 

OTP23 

-0.02866 

-0.12856 

-0.12765 

-0.02703 

0.01738 

-0.10052 

-0.10354 

0.15760 

0.15186 

0.7377 

0.1315 

0.1342 

0.7521 

0.8391 

0.2390 

0.2251 

0.0639 

0.0743 

ITS1 

-0.01304 

-0.03283 

-0.02277 

-0.00725 

0.00196 

-0.02617 

-0.02933 

0.40078 

0.39446 

0.8789 

0.7012 

0.7902 

0.9325 

0.9818 

0.7598 

0.7318 

<.0001 

<.0001 

OTS1 

-0.01535 

-0.02908 

-0.01834 

-0.00570 

0.00272 

-0.04078 

-0.04353 

0.46189 

0.45602 

0.8576 

0.7340 

0.8303 

0.9469 

0.9746 

0.6336 

0.6109 

<.0001 

<.0001 

ITS2 

0.00388 

-0.08192 

-0.07494 

-0.01808 

-0.00621 

0.69097 

0.69187 

-0.0451 1 

-0.04188 

0.9639 

0.3377 

0.3806 

0.8327 

0.9422 

<.0001 

<.0001 

0.5980 

0.6245 

OTS2 

0.01415 

-0.06497 

-0.05800 

-0.00002 

0.01335 

0.69667 

0.69838 

-0.03253 

-0.02874 

0.8687 

0.4473 

0.4976 

0.9999 

0.8760 

<.0001 

<.0001 

0.7038 

0.7370 

ITS5 

-0.02674 

-0.08192 

-0.07925 

-0.01808 

-0.04038 

-0.06529 

-0.04859 

-0.0451 1 

-0.03493 

0.7547 

0.3377 

0.3537 

0.8327 

0.6370 

0.4451 

0.5700 

0.5980 

0.6831 

OTS5 

-0.01821 

-0.08641 

-0.08389 

-0.02174 

-0.04515 

-0.06484 

-0.04782 

-0.04013 

-0.02968 

0.8315 

0.3118 

0.3262 

0.7995 

0.5977 

0.4482 

0.5762 

0.6391 

0.7287 

347 


Observed  to  Trained  NNS  Output  for  Exercises  1,2,3  18 

07:43  Wednesday,  January  15,  2003 

The  CORR  Procedure 

Pearson  Correlation  Coefficients,  N  =  139 
Prob  >  | r |  under  HO:  Rho=0 


ITA10 

0TA10 

ITA1 1 

OTA1 1 

ITA12 

0TA12 

ITA18 

0TA18 

ITA19 

OTP11 

-0.11028 

-0.13301 

-0.20244 

-0.20742 

-0.12307 

-0.09206 

0.19819 

0.19229 

-0.02758 

0.1962 

0.1185 

0.0168 

0.0143 

0.1489 

0.2811 

0.0193 

0.0233 

0.7472 

ITP12 

-0.03714 

-0.04176 

-0.07079 

-0.07181 

-0.04565 

-0.01934 

0.63202 

0.64432 

0.13062 

0.6642 

0.6254 

0.4076 

0.4009 

0.5936 

0.8212 

<.0001 

<.0001 

0.1254 

OTP12 

-0.04205 

-0.04712 

-0.07537 

-0.07691 

-0.03640 

-0.00898 

0.63967 

0.65225 

0.12888 

0.6231 

0.5818 

0.3779 

0.3682 

0.6706 

0.9164 

<.0001 

<.0001 

0.1305 

ITP15 

0.37616 

0.38135 

0.15200 

0.17072 

-0.04771 

-0.03482 

-0.07397 

-0.08900 

-0.04771 

<.0001 

<.0001 

0.0741 

0.0445 

0.5771 

0.6841 

0.3868 

0.2975 

0.5771 

OTP15 

0.37837 

0.38461 

0.15796 

0.17756 

-0.03325 

-0.01963 

-0.08055 

-0.09536 

-0.04388 

<.0001 

<.0001 

0.0633 

0.0365 

0.6975 

0.8186 

0.3459 

0.2641 

0.6080 

ITP22 

-0.02782 

-0.00748 

-0.05303 

-0.03666 

-0.03420 

-0.04448 

-0.05303 

-0.03477 

0.41857 

0.7451 

0.9304 

0.5353 

0.6683 

0.6894 

0.6031 

0.5353 

0.6845 

<.0001 

OTP22 

-0.01881 

0.00186 

-0.05014 

-0.03324 

-0.03758 

-0.04833 

-0.05295 

-0.03475 

0.41107 

0.8260 

0.9827 

0.5578 

0.6977 

0.6605 

0.5721 

0.5359 

0.6847 

<.0001 

ITP23 

-0.04202 

-0.05350 

0.34402 

0.35003 

-0.05166 

-0.04447 

0.02594 

0.04106 

-0.05166 

0.6233 

0.5317 

<.0001 

<.0001 

0.5459 

0.6032 

0.7618 

0.6313 

0.5459 

OTP23 

-0.04484 

-0.05584 

0.33899 

0.34594 

-0.03804 

-0.02990 

0.02247 

0.03855 

-0.03970 

0.6002 

0.5138 

<.0001 

<.0001 

0.6566 

0.7268 

0.7929 

0.6523 

0.6426 

ITS1 

-0.01029 

-0.00466 

-0.01960 

-0.01907 

-0.01264 

-0.01522 

-0.01960 

-0.02112 

-0.01264 

0.9043 

0.9565 

0.8188 

0.8237 

0.8826 

0.8589 

0.8188 

0.8050 

0.8826 

OTS1 

0.00652 

0.01316 

-0.01290 

-0.01023 

-0.00112 

-0.00274 

-0.03299 

-0.03272 

-0.00868 

0.9393 

0.8778 

0.8802 

0.9048 

0.9896 

0 . 9744 

0.6998 

0.7022 

0.9192 

ITS2 

-0.02566 

-0.00365 

-0.04891 

-0.04381 

-0.03155 

-0.03643 

-0.04891 

-0.05390 

-0.03155 

0 . 7643 

0.9660 

0.5675 

0.6086 

0.7124 

0.6702 

0.5675 

0.5286 

0.7124 

OTS2 

-0.02590 

-0.00279 

-0.04232 

-0.03613 

-0.03537 

-0.03972 

-0.04651 

-0.05077 

-0.01921 

0.7622 

0.9740 

0.6209 

0.6728 

0.6793 

0.6425 

0.5867 

0.5528 

0.8224 

ITS5 

-0.02566 

-0.00611 

0.27484 

0.27635 

-0.03155 

-0.04128 

-0.04891 

-0.05578 

-0.03155 

0.7643 

0.9431 

0.0011 

0.0010 

0.7124 

0.6295 

0.5675 

0.5143 

0.7124 

OTS5 

-0.01302 

0.00646 

0.27032 

0.27271 

-0.03767 

-0.04768 

-0.05864 

-0.06529 

-0.02768 

0.8790 

0.9398 

0.0013 

0.0012 

0.6597 

0.5773 

0.4929 

0.4451 

0 . 7464 

348 


Observed  to  Trained  NNS  Output  for  Exercises  1,2,3  19 

07:43  Wednesday,  January  15,  2003 

The  CORR  Procedure 

Pearson  Correlation  Coefficients,  N  =  139 
Prob  >  | r |  under  HO:  Rho=0 


0TA19 

ITA21 

0TA21 

ITA23 

OTA23 

IM1 

0M1 

IM2 

0M2 

OTP1 1 

-0.04284 

0.49863 

0.49441 

0.45576 

0.46819 

0.11369 

0.09697 

-0.07614 

-0.06492 

0.6165 

<.0001 

<■0001 

<.0001 

<■0001 

0.1826 

0.2561 

0.3730 

0.4477 

ITP12 

0.14164 

-0.13329 

-0.13363 

0.10807 

0.11436 

0.11456 

0.11403 

-0.08101 

-0.07939 

0.0963 

0.1177 

0.1168 

0.2054 

0.1801 

0.1793 

0.1814 

0.3431 

0.3529 

OTP12 

0.13933 

-0.12304 

-0.12352 

0.11412 

0.12094 

0.11384 

0.11307 

-0.08561 

-0.08389 

0.1019 

0.1490 

0.1474 

0.1810 

0.1561 

0.1821 

0.1851 

0.3163 

0.3262 

ITP15 

-0.07012 

-0.13929 

-0.14424 

0.40438 

0.42087 

-0.16341 

-0.17044 

-0.34157 

-0.32634 

0.4121 

0.1020 

0.0903 

<.0001 

<.0001 

0.0546 

0.0449 

<.0001 

<.0001 

OTP15 

-0.06661 

-0.15175 

-0.15671 

0.41657 

0.43307 

-0.16848 

-0.17523 

-0.34973 

-0.33504 

0.4359 

0.0745 

0.0654 

<.0001 

<.0001 

0 . 0474 

0.0391 

<.0001 

<.0001 

ITP22 

0.42976 

-0.09986 

-0.10562 

-0.11307 

-0.11639 

0.32028 

0.33523 

-0.24488 

-0.25447 

<.0001 

0.2422 

0.2159 

0.1851 

0.1724 

0.0001 

<.0001 

0.0037 

0.0025 

OTP22 

0.42370 

-0.11467 

-0.12064 

-0.11930 

-0.12293 

0.32660 

0.3421 1 

-0.25571 

-0.26610 

<.0001 

0.1789 

0.1572 

0.1619 

0.1494 

<.0001 

<.0001 

0.0024 

0.0015 

ITP23 

-0.01821 

-0.15082 

-0.15324 

-0.17077 

-0.16476 

0.12436 

0.13802 

-0.36985 

-0.38208 

0.8315 

0.0764 

0.0717 

0 . 0444 

0.0526 

0.1447 

0.1052 

<.0001 

<.0001 

OTP23 

-0.00709 

-0.16012 

-0.16281 

-0.16047 

-0.15458 

0.13814 

0.15197 

-0.37864 

-0.39128 

0.9340 

0.0597 

0.0555 

0.0592 

0.0692 

0.1049 

0.0741 

<.0001 

<.0001 

ITS1 

-0.01849 

-0.03691 

-0.03707 

-0.04180 

-0.04191 

0.18822 

0.18492 

-0.09052 

-0.09474 

0.8290 

0.6662 

0.6648 

0.6252 

0.6243 

0.0265 

0.0293 

0.2892 

0.2673 

OTS1 

-0.01466 

-0.04356 

-0.04465 

-0.04396 

-0.04365 

0.21460 

0.21288 

-0.10116 

-0.10501 

0.8640 

0.6106 

0.6017 

0.6073 

0.6099 

0.0112 

0.0119 

0.2360 

0.2186 

ITS2 

-0.02483 

-0.09210 

-0.08674 

-0.10429 

-0.10400 

0.08450 

0.09017 

-0.22586 

-0.23478 

0.7717 

0.2809 

0.3100 

0.2218 

0.2231 

0.3226 

0.2912 

0.0075 

0.0054 

OTS2 

-0.01116 

-0.09154 

-0.08653 

-0.10719 

-0.10679 

0.10121 

0.10755 

-0.24373 

-0.25258 

0.8962 

0.2838 

0.3111 

0.2091 

0.2108 

0.2358 

0.2076 

0.0038 

0.0027 

ITS5 

-0.03081 

-0.09210 

-0.09915 

0.25363 

0.25498 

0.21288 

0.20448 

-0.12393 

-0.11886 

0.7188 

0.2809 

0.2455 

0.0026 

0.0025 

0.0119 

0.0158 

0.1461 

0.1634 

OTS5 

-0.02679 

-0.08053 

-0.08794 

0.26080 

0.26187 

0.21200 

0.20347 

-0.11782 

-0.11104 

0.7542 

0.3460 

0.3033 

0.0019 

0.0018 

0.0122 

0.0163 

0.1672 

0.1931 
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Observed  to  Trained  NNS  Output  for  Exercises  1,2,3 

07:43  Wednesday,  January  15, 

The  CORR  Procedure 

Pearson  Correlation  Coefficients,  N  =  139 
Prob  >  | r |  under  HO:  Rho=0 


IM3 

0M3 

IM4 

0M4 

IM5 

0M5 

OTP1 1 

0.21655 

0.22033 

-0.11028 

-0.16731 

0.13747 

0.14753 

0.0105 

0.0092 

0.1962 

0.0490 

0.1066 

0.0831 

ITP12 

0.05643 

0.05777 

-0.03714 

-0.04488 

-0.10093 

-0.10461 

0.5094 

0.4994 

0.6642 

0.5999 

0.2371 

0.2204 

OTP12 

0.06014 

0.06122 

-0.04205 

-0.05127 

-0.09762 

-0.10158 

0.4819 

0.4741 

0.6231 

0.5489 

0.2529 

0.2341 

ITP15 

0.66723 

0.67079 

0.37616 

0.37133 

0.44686 

0.45886 

<.0001 

<.0001 

<.0001 

<.0001 

<.0001 

<.0001 

OTP15 

0.67870 

0.68346 

0.37837 

0.37528 

0.45410 

0.46644 

<.0001 

<.0001 

<.0001 

<.0001 

<.0001 

<■0001 

ITP22 

0.03858 

0.04929 

-0.02782 

-0.06281 

-0.07561 

-0.07107 

0.6521 

0.5645 

0.7451 

0.4626 

0.3763 

0.4058 

OTP22 

0.03259 

0.04365 

-0.01881 

-0.05400 

-0.07461 

-0.07019 

0.7033 

0.6099 

0.8260 

0.5278 

0.3827 

0.4116 

ITP23 

-0.13163 

-0.12607 

-0.04202 

-0.06793 

-0.11420 

-0.11882 

0.1224 

0.1392 

0.6233 

0.4269 

0.1807 

0.1636 

OTP23 

-0.13598 

-0.12950 

-0.04484 

-0.06992 

-0.11936 

-0.12378 

0.1105 

0.1287 

0.6002 

0.4134 

0.1616 

0.1466 

ITS1 

-0.03222 

-0.01696 

-0.01029 

-0.02747 

-0.02795 

-0.01463 

0.7065 

0.8429 

0.9043 

0.7482 

0 . 7440 

0.8643 

OTS1 

-0.04447 

-0.02743 

0.00652 

-0.01459 

-0.04499 

-0.03056 

0.6032 

0.7486 

0.9393 

0.8647 

0.5989 

0.7210 

ITS2 

-0.08039 

-0.08720 

-0.02566 

-0.01905 

0.52375 

0.53703 

0.3469 

0.3074 

0 . 7643 

0.8239 

<.0001 

<.0001 

OTS2 

-0.08270 

-0.08893 

-0.02590 

-0.01882 

0.52285 

0.53640 

0.3331 

0.2979 

0.7622 

0.8259 

<.0001 

<.0001 

ITS5 

0.19020 

0.20511 

-0.02566 

0.03622 

-0.06974 

-0.05903 

0.0249 

0.0154 

0.7643 

0.6721 

0.4146 

0.4900 

OTS5 

0.19385 

0.20878 

-0.01302 

0.04953 

-0.06895 

-0.05732 

0.0222 

0.0136 

0.8790 

0.5625 

0.4199 

0.5027 

20 

2003 
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Observed  to  Trained  NNS  Output  for  Exercises  1,2,3  21 

07:43  Wednesday,  January  15,  2003 

The  CORR  Procedure 

Pearson  Correlation  Coefficients,  N  =  139 
Prob  >  | r |  under  HO:  Rho=0 


ITP2 

OTP2 

ITP3 

OTP3 

ITP4 

0TP4 

ITP5 

OTP5 

ITP7 

ITS8 

-0.00725 

-0.03288 

-0.01808 

-0.01843 

-0.00725 

0.09287 

0.40078 

0.39867 

-0.02104 

0.9325 

0.7008 

0.8327 

0.8295 

0.9325 

0.2769 

<.0001 

<.0001 

0.8058 

0TS8 

-0.00325 

0.24461 

0.40703 

0.42105 

-0.04592 

-0.06368 

0.31478 

0.33548 

-0.11781 

0.9697 

0.0037 

<.0001 

<.0001 

0.5914 

0.4564 

0.0002 

<.0001 

0.1672 

ITS1 1 

-0.01644 

0.44504 

0.52926 

0.52643 

-0.01644 

-0.00341 

0.52926 

0.53133 

-0.04774 

0.8476 

<.0001 

<.0001 

<.0001 

0.8476 

0.9682 

<.0001 

<.0001 

0.5768 

OTS11 

-0.02320 

0.46239 

0.53223 

0.53092 

-0.00725 

0.00622 

0.54912 

0.55314 

-0.06315 

0.7863 

<.0001 

<.0001 

<.0001 

0.9325 

0.9420 

<.0001 

<.0001 

0.4602 

ITS12 

-0.04275 

-0.11163 

-0.10668 

-0.11876 

-0.04275 

-0.02837 

-0.10668 

-0.11036 

0.02992 

0.6173 

0.1908 

0.2113 

0.1638 

0.6173 

0.7402 

0.2113 

0.1959 

0.7266 

OTS12 

-0.03941 

-0.11793 

-0.10517 

-0.11722 

-0.04186 

-0.02631 

-0.10822 

-0.11165 

0.02492 

0.6450 

0.1668 

0.2179 

0.1694 

0.6247 

0.7585 

0.2048 

0.1907 

0.7709 

ITS15 

-0.01465 

0.04645 

-0.03656 

-0.01683 

-0.01465 

-0.00956 

0.17519 

0.19556 

-0.04254 

0.8641 

0.5871 

0.6692 

0.8441 

0.8641 

0.9111 

0.0391 

0.0210 

0.6190 

OTS15 

-0.01709 

0.07270 

-0.01869 

0.00124 

-0.00322 

0.00679 

0.20176 

0.22374 

-0.05096 

0.8417 

0.3950 

0.8272 

0.9885 

0.9700 

0.9368 

0.0172 

0.0081 

0.5513 

ITS18 

0.25436 

0.25239 

0.28179 

0.29556 

-0.02849 

-0.03559 

-0.07108 

-0.06231 

-0.08270 

0.0025 

0.0027 

0.0008 

0.0004 

0.7392 

0.6775 

0.4057 

0.4662 

0.3331 

OTS18 

0.25053 

0.25693 

0.28246 

0.29705 

-0.02766 

-0.03467 

-0.06914 

-0.05935 

-0.08395 

0.0029 

0.0023 

0.0008 

0.0004 

0.7465 

0.6853 

0.4186 

0.4877 

0.3258 

ITS19 

-0.02617 

-0.14956 

-0.06529 

-0.05916 

-0.02617 

-0.04738 

-0.06529 

-0.07233 

-0.07596 

0.7598 

0.0789 

0.4451 

0.4891 

0.7598 

0.5797 

0.4451 

0.3975 

0.3741 

OTS19 

-0.02917 

-0.15260 

-0.05564 

-0.04984 

-0.02949 

-0.05147 

-0.06105 

-0.06891 

-0.07917 

0.7332 

0.0729 

0.5153 

0.5601 

0.7304 

0 . 5474 

0.4753 

0.4202 

0.3542 

ITS20 

-0.01029 

0.03261 

-0.02566 

-0.02554 

-0.01029 

0.02180 

-0.02566 

-0.01918 

-0.02986 

0 . 9043 

0.7032 

0 . 7643 

0.7654 

0 . 9043 

0.7989 

0 . 7643 

0.8227 

0.7271 

0TS20 

-0.00454 

0.04463 

-0.03944 

-0.04066 

0.01931 

0.05759 

-0.00902 

-0.00293 

0.0021 1 

0.9577 

0.6019 

0.6448 

0.6346 

0.8215 

0.5007 

0.9161 

0.9726 

0.9804 

ITS22 

-0.01029 

0.01279 

-0.02566 

-0.02772 

-0.01029 

0.02260 

0.27160 

0.28086 

-0.02986 

0.9043 

0.8813 

0 . 7643 

0.7460 

0 . 9043 

0.7917 

0.0012 

0.0008 

0.7271 
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Observed  to  Trained  NNS  Output  for  Exercises  1,2,3  22 

07:43  Wednesday,  January  15,  2003 

The  CORR  Procedure 

Pearson  Correlation  Coefficients,  N  =  139 
Prob  >  | r |  under  HO:  Rho=0 


0TP7 

ITP8 

0TP8 

ITP10 

0TP10 

ITP1 1 

OTP1 1 

ITP12 

0TP12 

ITS8 

-0.01740 

-0.01029 

-0.01521 

-0.02961 

-0.03151 

-0.07419 

-0.06928 

-0.02617 

-0.02963 

0.8389 

0.9043 

0.8589 

0.7294 

0.7127 

0.3854 

0.4177 

0.7598 

0.7292 

0TS8 

-0.11295 

-0.03490 

-0.04860 

0.08665 

0.09610 

0.18362 

0.18595 

0.49150 

0.50524 

0.1855 

0.6834 

0.5699 

0.3105 

0.2604 

0.0305 

0.0284 

<.0001 

<.0001 

ITS1 1 

-0.05106 

-0.02334 

-0.03871 

-0.06718 

-0.05200 

-0.16834 

-0.16656 

-0.05938 

-0.05529 

0.5506 

0.7851 

0.6510 

0.4320 

0.5432 

0.0476 

0.0500 

0.4875 

0.5180 

OTS11 

-0.06678 

-0.03294 

-0.04955 

-0.05003 

-0.03418 

-0.16934 

-0.16791 

-0.04233 

-0.03800 

0.4348 

0.7003 

0.5624 

0.5586 

0.6896 

0.0463 

0.0482 

0.6208 

0.6570 

ITS12 

0.03786 

0.24057 

0.25610 

-0.05906 

-0.05894 

0.43145 

0.43666 

-0.09052 

-0.07653 

0.6581 

0 . 0043 

0.0023 

0.4898 

0.4907 

<.0001 

<.0001 

0.2893 

0.3706 

OTS12 

0.03352 

0.23909 

0.25514 

-0.05759 

-0.05750 

0.44024 

0.44566 

-0.08622 

-0.07197 

0.6952 

0.0046 

0.0024 

0.5007 

0.5014 

<.0001 

<.0001 

0.3129 

0.3998 

ITS15 

-0.03595 

-0.02080 

-0.02377 

0.35622 

0.35459 

0.11063 

0.11270 

0.40676 

0.39927 

0.6744 

0.8080 

0.7812 

<.0001 

<.0001 

0.1948 

0.1865 

<.0001 

<.0001 

OTS15 

-0.04261 

-0.02312 

-0.02528 

0.38893 

0.38835 

0.12989 

0.13275 

0.43082 

0.42460 

0.6185 

0.7870 

0.7677 

<.0001 

<.0001 

0.1275 

0.1193 

<.0001 

<.0001 

ITS18 

-0.07492 

-0.04044 

-0.01903 

0.57700 

0.59005 

0.38401 

0.38154 

0.06736 

0.07312 

0.3807 

0.6365 

0.8241 

<.0001 

<.0001 

<.0001 

<.0001 

0.4308 

0.3923 

OTS18 

-0.07575 

-0.03842 

-0.01660 

0.59381 

0.60711 

0.39031 

0.38810 

0.08780 

0.09375 

0.3755 

0.6534 

0.8462 

<.0001 

<.0001 

<.0001 

<.0001 

0.3041 

0.2723 

ITS19 

-0.08974 

-0.03714 

-0.05956 

-0.10691 

-0.09804 

-0.11274 

-0.11958 

-0.09449 

-0.09602 

0.2934 

0.6642 

0.4861 

0.2103 

0.2509 

0.1864 

0.1609 

0.2685 

0.2608 

OTS19 

-0.09378 

-0.04231 

-0.06558 

-0.11972 

-0.11073 

-0.11478 

-0.12173 

-0.10163 

-0.10303 

0.2722 

0.6209 

0.4431 

0.1604 

0.1944 

0.1785 

0.1534 

0.2339 

0.2274 

ITS20 

-0.01497 

-0.01460 

0.00840 

-0.04202 

-0.04512 

-0.10530 

-0.10634 

-0.03714 

-0.04250 

0.8611 

0.8646 

0.9218 

0.6233 

0.5979 

0.2173 

0.2128 

0.6642 

0.6194 

0TS20 

0.01774 

0.00653 

0.02994 

-0.06187 

-0.06502 

-0.09846 

-0.09934 

-0.05349 

-0.05865 

0.8358 

0.9392 

0.7264 

0.4693 

0.4470 

0.2488 

0.2446 

0.5317 

0.4928 

ITS22 

-0.01119 

-0.01460 

0.01113 

0.15268 

0.15062 

0.01667 

0.01932 

-0.03714 

-0.03193 

0.8960 

0.8646 

0.8966 

0.0727 

0.0767 

0.8456 

0.8214 

0.6642 

0.7090 
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Observed  to  Trained  NNS  Output  for  Exercises  1,2,3  23 

07:43  Wednesday,  January  15,  2003 

The  CORR  Procedure 

Pearson  Correlation  Coefficients,  N  =  139 
Prob  >  | r |  under  HO:  Rho=0 


ITP15 

0TP15 

ITP22 

0TP22 

ITP23 

OTP23 

ITS1 

OTS1 

ITS2 

ITS8 

-0.02734 

-0.02951 

-0.01960 

-0.02275 

-0.02961 

-0.02560 

-0.00725 

-0.00895 

-0.01808 

0.7493 

0.7302 

0.8188 

0.7904 

0.7294 

0.7648 

0.9325 

0.9168 

0.8327 

0TS8 

-0.06429 

-0.06000 

-0.14070 

-0.14998 

-0.12718 

-0.11891 

-0.02885 

-0.03029 

-0.11457 

0.4521 

0.4829 

0.0985 

0.0780 

0.1357 

0.1633 

0.7360 

0.7234 

0.1793 

ITS1 1 

0.20332 

0.21602 

-0.04448 

-0.04135 

-0.06718 

-0.05836 

-0.01644 

0.00330 

-0.04103 

0.0164 

0.0106 

0.6031 

0.6289 

0.4320 

0.4950 

0.8476 

0.9693 

0.6315 

OTS11 

0.22105 

0.23475 

-0.04876 

-0.0461 1 

-0.04937 

-0.04030 

-0.00580 

0.01448 

-0.05307 

0.0089 

0.0054 

0.5687 

0.5899 

0.5638 

0.6376 

0.9460 

0.8656 

0.5349 

ITS12 

-0.16133 

-0.16335 

-0.11566 

-0.12715 

0.05656 

0.05039 

-0.04275 

-0.04251 

-0.10668 

0.0578 

0.0547 

0.1751 

0.1358 

0.5084 

0.5558 

0.6173 

0.6193 

0.2113 

OTS12 

-0.16225 

-0.16445 

-0.11692 

-0.12850 

0.05413 

0.04788 

-0.04274 

-0.04193 

-0.11154 

0.0564 

0.0531 

0.1705 

0.1317 

0.5268 

0.5757 

0.6173 

0.6241 

0.1911 

ITS15 

-0.05529 

-0.03798 

-0.03964 

-0.04600 

-0.05987 

-0.05927 

-0.01465 

-0.0241 1 

-0.03656 

0.5180 

0.6571 

0.6431 

0.5908 

0.4839 

0.4883 

0.8641 

0.7781 

0.6692 

OTS15 

-0.02238 

-0.00463 

-0.04829 

-0.05568 

-0.07578 

-0.07490 

-0.01816 

-0.02825 

-0.04486 

0.7937 

0.9569 

0.5724 

0.5150 

0.3753 

0.3808 

0.8320 

0.7413 

0.6000 

ITS18 

0.63140 

0.62909 

-0.07707 

-0.08104 

-0.11640 

-0.10970 

-0.02849 

-0.02300 

-0.07108 

<.0001 

<.0001 

0.3672 

0.3430 

0.1724 

0.1986 

0.7392 

0.7881 

0.4057 

OTS18 

0.63125 

0.62985 

-0.08065 

-0.08476 

-0.10771 

-0.10096 

-0.02825 

-0.02301 

-0.07369 

<.0001 

<.0001 

0.3453 

0.3211 

0.2069 

0.2370 

0.7413 

0.7880 

0.3886 

ITS19 

-0.09874 

-0.09576 

-0.07079 

-0.06135 

0.63612 

0.63910 

-0.02617 

-0.03752 

-0.06529 

0.2475 

0.2621 

0.4076 

0.4731 

<.0001 

<.0001 

0.7598 

0.6610 

0.4451 

OTS19 

-0.11064 

-0.10782 

-0.05611 

-0.04654 

0.63971 

0.64333 

-0.02854 

-0.03913 

-0.06593 

0.1948 

0.2064 

0.5118 

0.5865 

<.0001 

<.0001 

0.7387 

0 . 6474 

0.4406 

ITS20 

-0.03881 

-0.04275 

-0.02782 

-0.00533 

0.34739 

0.34023 

-0.01029 

0.06648 

-0.02566 

0.6501 

0.6173 

0.7451 

0.9503 

<.0001 

<.0001 

0 . 9043 

0.4368 

0 . 7643 

0TS20 

-0.05822 

-0.06239 

0.01076 

0.03139 

0.36068 

0.35564 

0.07974 

0.15366 

-0.00141 

0.4960 

0.4656 

0.9000 

0.7138 

<.0001 

<.0001 

0.3507 

0.0709 

0.9868 

ITS22 

-0.03881 

-0.03065 

-0.02782 

-0.02369 

-0.04202 

-0.03998 

-0.01029 

-0.01270 

-0.02566 

0.6501 

0.7202 

0.7451 

0.7820 

0.6233 

0 . 6403 

0 . 9043 

0.8821 

0 . 7643 
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The  CORR  Procedure 

Pearson  Correlation  Coefficients,  N  =  139 
Prob  >  | r |  under  HO:  Rho=0 


OTS2 

ITS5 

OTS5 

ITS8 

0TS8 

ITS1 1 

0TS11 

ITS12 

0TS12 

ITS8 

-0.01130 

-0.01808 

-0.00014 

1 .00000 

-0.04592 

-0.01644 

-0.00048 

-0.04275 

-0.03475 

0.8950 

0.8327 

0.9987 

0.5914 

0.8476 

0.9955 

0.6173 

0.6846 

0TS8 

-0.11766 

-0.12876 

-0.14173 

-0.04592 

1 .00000 

0.36050 

0.38931 

-0.26012 

-0.26099 

0.1677 

0.1309 

0.0960 

0.5914 

<.0001 

<.0001 

0.0020 

0.0019 

ITS1 1 

-0.04416 

-0.04103 

-0.03773 

-0.01644 

0.36050 

1 .00000 

0.99624 

-0.09702 

-0.09891 

0.6057 

0.6315 

0.6592 

0.8476 

<.0001 

<.0001 

0.2559 

0.2467 

OTS11 

-0.05587 

-0.03880 

-0.03576 

-0.00048 

0.38931 

0.99624 

1 .00000 

-0.11928 

-0.12146 

0.5136 

0.6502 

0.6760 

0.9955 

<.0001 

<.0001 

0.1619 

0.1544 

ITS12 

-0.10702 

-0.10668 

-0.10077 

-0.04275 

-0.26012 

-0.09702 

-0.11928 

1 .00000 

0.99904 

0.2099 

0.2113 

0.2379 

0.6173 

0.0020 

0.2559 

0.1619 

<.0001 

OTS12 

-0.11182 

-0.11200 

-0.10587 

-0.03475 

-0.26099 

-0.09891 

-0.12146 

0.99904 

1 .00000 

0.1900 

0.1893 

0.2148 

0.6846 

0.0019 

0.2467 

0.1544 

<.0001 

ITS15 

-0.04107 

-0.03656 

-0.03259 

-0.01465 

0.18321 

-0.03325 

-0.01588 

-0.08645 

-0.08318 

0.6312 

0.6692 

0.7033 

0.8641 

0.0309 

0.6976 

0.8528 

0.3116 

0.3303 

OTS15 

-0.04939 

-0.01691 

-0.01315 

0.00798 

0.21938 

-0.01361 

0.00579 

-0.09161 

-0.08858 

0.5637 

0.8433 

0.8779 

0.9257 

0.0095 

0.8737 

0.9461 

0.2835 

0.2998 

ITS18 

-0.07308 

-0.07108 

-0.07383 

-0.02849 

0.02794 

-0.06465 

-0.04982 

-0.16808 

-0.16343 

0.3926 

0.4057 

0.3877 

0.7392 

0.7440 

0.4496 

0.5603 

0.0479 

0.0546 

OTS18 

-0.07577 

-0.07393 

-0.07700 

-0.02943 

0.03989 

-0.06384 

-0.04787 

-0.16906 

-0.16425 

0.3753 

0.3871 

0.3676 

0.7309 

0.6410 

0.4553 

0.5758 

0.0466 

0.0533 

ITS19 

-0.05505 

-0.06529 

-0.07015 

-0.02617 

-0.12216 

-0.05938 

-0.04335 

-0.15439 

-0.15127 

0.5198 

0.4451 

0.4119 

0.7598 

0.1520 

0.4875 

0.6124 

0.0696 

0.0755 

OTS19 

-0.05506 

-0.06105 

-0.06640 

-0.02981 

-0.13051 

-0.05324 

-0.03845 

-0.14191 

-0.13952 

0.5197 

0.4753 

0.4374 

0.7276 

0.1257 

0.5336 

0.6531 

0.0956 

0.1014 

ITS20 

-0.01604 

-0.02566 

-0.01647 

-0.01029 

-0.05912 

-0.02334 

-0.03191 

-0.06068 

-0.05279 

0.8513 

0 . 7643 

0.8474 

0 . 9043 

0.4894 

0.7851 

0.7092 

0.4779 

0.5371 

0TS20 

0.00981 

-0.00836 

0.00250 

0.01295 

-0.10248 

-0.04170 

-0.05069 

-0.01356 

-0.00510 

0.9088 

0.9222 

0.9767 

0.8797 

0.2300 

0.6260 

0.5535 

0.8741 

0.9525 

ITS22 

-0.01727 

-0.02566 

-0.01647 

-0.01029 

-0.06517 

-0.02334 

-0.01338 

-0.06068 

-0.05421 

0.8401 

0 . 7643 

0.8474 

0 . 9043 

0.4459 

0.7851 

0.8758 

0.4779 

0.5262 

354 


Observed  to  Trained  NNS  Output  for  Exercises  1,2,3  25 

07:43  Wednesday,  January  15,  2003 
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Pearson  Correlation  Coefficients,  N  =  139 
Prob  >  | r |  under  HO:  Rho=0 


ITS15 

0TS15 

ITS18 

0TS18 

ITS19 

0TS19 

ITS20 

0TS20 

ITS22 

ITS8 

-0.01465 

0.00798 

-0.02849 

-0.02943 

-0.02617 

-0.02981 

-0.01029 

0.01295 

-0.01029 

0.8641 

0.9257 

0.7392 

0.7309 

0.7598 

0.7276 

0.9043 

0.8797 

0.9043 

0TS8 

0.18321 

0.21938 

0.02794 

0.03989 

-0.12216 

-0.13051 

-0.05912 

-0.10248 

-0.06517 

0.0309 

0.0095 

0 . 7440 

0.6410 

0.1520 

0.1257 

0.4894 

0.2300 

0.4459 

ITS1 1 

-0.03325 

-0.01361 

-0.06465 

-0.06384 

-0.05938 

-0.05324 

-0.02334 

-0.04170 

-0.02334 

0.6976 

0.8737 

0.4496 

0.4553 

0.4875 

0.5336 

0.7851 

0.6260 

0.7851 

OTS11 

-0.01588 

0.00579 

-0.04982 

-0.04787 

-0.04335 

-0.03845 

-0.03191 

-0.05069 

-0.01338 

0.8528 

0.9461 

0.5603 

0.5758 

0.6124 

0.6531 

0.7092 

0.5535 

0.8758 

ITS12 

-0.08645 

-0.09161 

-0.16808 

-0.16906 

-0.15439 

-0.14191 

-0.06068 

-0.01356 

-0.06068 

0.3116 

0.2835 

0.0479 

0.0466 

0.0696 

0.0956 

0.4779 

0.8741 

0.4779 

OTS12 

-0.08318 

-0.08858 

-0.16343 

-0.16425 

-0.15127 

-0.13952 

-0.05279 

-0.00510 

-0.05421 

0.3303 

0.2998 

0.0546 

0.0533 

0.0755 

0.1014 

0.5371 

0.9525 

0.5262 

ITS15 

1  .00000 

0.99642 

-0.05761 

-0.04131 

-0.05291 

-0.05980 

-0.02080 

-0.02567 

-0.02080 

<.0001 

0.5006 

0.6292 

0.5362 

0.4844 

0.8080 

0 . 7643 

0.8080 

OTS15 

0.99642 

1 .00000 

-0.04053 

-0.02353 

-0.06701 

-0.07491 

-0.02539 

-0.03142 

-0.01479 

<.0001 

0.6357 

0.7833 

0.4332 

0.3808 

0.7667 

0.7135 

0.8628 

ITS18 

-0.05761 

-0.04053 

1 .00000 

0.99919 

-0.10287 

-0.10677 

-0.04044 

-0.04800 

-0.04044 

0.5006 

0.6357 

<.0001 

0.2282 

0.2109 

0.6365 

0.5747 

0.6365 

OTS18 

-0.04131 

-0.02353 

0.99919 

1 .00000 

-0.09713 

-0.10137 

-0.04178 

-0.04976 

-0.02962 

0.6292 

0.7833 

<.0001 

0.2553 

0.2351 

0.6253 

0.5607 

0.7292 

ITS19 

-0.05291 

-0.06701 

-0.10287 

-0.09713 

1 .00000 

0.99836 

-0.03714 

-0.03770 

-0.03714 

0.5362 

0.4332 

0.2282 

0.2553 

<.0001 

0.6642 

0.6595 

0.6642 

OTS19 

-0.05980 

-0.07491 

-0.10677 

-0.10137 

0.99836 

1 .00000 

-0.03961 

-0.03918 

-0.04164 

0.4844 

0.3808 

0.2109 

0.2351 

<.0001 

0 . 6434 

0.6470 

0.6265 

ITS20 

-0.02080 

-0.02539 

-0.04044 

-0.04178 

-0.03714 

-0.03961 

1 .00000 

0.98841 

-0.01460 

0.8080 

0.7667 

0.6365 

0.6253 

0.6642 

0 . 6434 

<.0001 

0.8646 

0TS20 

-0.02567 

-0.03142 

-0.04800 

-0.04976 

-0.03770 

-0.03918 

0.98841 

1 .00000 

0.00597 

0 . 7643 

0.7135 

0.5747 

0.5607 

0.6595 

0.6470 

<.0001 

0.9444 

ITS22 

-0.02080 

-0.01479 

-0.04044 

-0.02962 

-0.03714 

-0.04164 

-0.01460 

0.00597 

1 .00000 

0.8080 

0.8628 

0.6365 

0.7292 

0.6642 

0.6265 

0.8646 

0 . 9444 
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Prob  >  | r |  under  HO:  Rho=0 


0TS22 

ITS23 

0TS23 

ITA2 

OTA2 

ITA3 

OTA3 

ITA7 

0TA7 

ITS8 

0.03811 

-0.03283 

-0.03579 

-0.00725 

-0.01832 

-0.02617 

-0.01605 

-0.01808 

-0.00104 

0.6561 

0.7012 

0.6757 

0.9325 

0.8305 

0.7598 

0.8513 

0.8327 

0.9903 

0TS8 

-0.05492 

0.42770 

0.43397 

-0.06298 

-0.10011 

0.17312 

0.17259 

-0.11457 

-0.13544 

0.5208 

<■0001 

<.0001 

0.4614 

0.2410 

0.0415 

0.0422 

0.1793 

0.1119 

ITS1 1 

-0.00181 

-0.07450 

-0.05831 

-0.01644 

-0.01880 

-0.05938 

-0.05019 

-0.04103 

-0.05121 

0.9832 

0.3834 

0.4953 

0.8476 

0.8261 

0.4875 

0.5573 

0.6315 

0.5493 

OTS11 

0.01043 

-0.06145 

-0.04527 

-0.01789 

-0.02060 

-0.06051 

-0.05130 

-0.04443 

-0.05600 

0.9030 

0.4724 

0.5967 

0.8345 

0.8098 

0.4792 

0.5487 

0.6035 

0.5126 

ITS12 

-0.06502 

-0.19371 

-0.20340 

-0.04275 

-0.05816 

-0.15439 

-0.15233 

-0.10668 

-0.10307 

0.4470 

0.0223 

0.0163 

0.6173 

0.4965 

0.0696 

0.0734 

0.2113 

0.2273 

OTS12 

-0.05922 

-0.19501 

-0.20512 

-0.04452 

-0.06023 

-0.15956 

-0.15777 

-0.10563 

-0.10205 

0.4887 

0.0214 

0.0154 

0.6028 

0.4812 

0.0606 

0.0636 

0.2159 

0.2319 

ITS15 

-0.01305 

-0.06639 

-0.06351 

-0.01465 

-0.03704 

-0.05291 

-0.05116 

-0.03656 

-0.04608 

0.8788 

0.4374 

0.4576 

0.8641 

0.6651 

0.5362 

0.5498 

0.6692 

0.5901 

OTS15 

-0.00437 

-0.05206 

-0.04955 

-0.01869 

-0.04477 

-0.05513 

-0.05256 

-0.04574 

-0.05616 

0.9592 

0.5428 

0.5624 

0.8271 

0.6007 

0.5192 

0.5389 

0.5928 

0.5114 

ITS18 

-0.01166 

-0.12908 

-0.13430 

-0.02849 

-0.03216 

-0.10287 

-0.11069 

-0.07108 

-0.08219 

0.8916 

0.1299 

0.1150 

0.7392 

0.7070 

0.2282 

0.1945 

0.4057 

0.3361 

OTS18 

-0.00009 

-0.12704 

-0.13237 

-0.02943 

-0.03288 

-0.10797 

-0.11585 

-0.07406 

-0.08588 

0.9991 

0.1361 

0.1203 

0.7309 

0.7008 

0.2058 

0.1745 

0.3863 

0.3148 

ITS19 

-0.02750 

-0.11856 

-0.12326 

-0.02617 

0.02280 

-0.09449 

-0.09763 

-0.06529 

-0.06946 

0.7479 

0.1645 

0.1483 

0.7598 

0.7899 

0.2685 

0.2529 

0.4451 

0.4165 

OTS19 

-0.03144 

-0.11909 

-0.12387 

-0.01394 

0.03688 

-0.10144 

-0.10435 

-0.05524 

-0.05877 

0.7133 

0.1626 

0.1463 

0.8706 

0.6665 

0.2348 

0.2215 

0.5183 

0.4920 

ITS20 

-0.02346 

-0.04660 

-0.04741 

-0.01029 

-0.02146 

-0.03714 

-0.04163 

0.56886 

0.56923 

0.7840 

0.5859 

0.5794 

0 . 9043 

0.8020 

0.6642 

0.6265 

<.0001 

<.0001 

0TS20 

0.00009 

-0.06781 

-0.06761 

-0.01011 

-0.02008 

-0.03506 

-0.03838 

0.59543 

0.59775 

0.9992 

0.4276 

0.4291 

0.9060 

0.8145 

0.6820 

0.6537 

<.0001 

<.0001 

ITS22 

0.99121 

-0.04660 

-0.04496 

-0.01029 

-0.02373 

-0.03714 

-0.03580 

-0.02566 

-0.01956 

<.0001 

0.5859 

0.5992 

0 . 9043 

0.7816 

0.6642 

0.6757 

0 . 7643 

0.8192 
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Pearson  Correlation  Coefficients,  N  =  139 
Prob  >  | r |  under  HO:  Rho=0 


ITA10 

0TA10 

ITA1 1 

0TA1 1 

ITA12 

OTA12 

ITA18 

0TA18 

ITA19 

ITS8 

-0.01029 

0.00495 

-0.01960 

-0.00510 

-0.01264 

0.00931 

-0.01960 

-0.02359 

-0.01264 

0.9043 

0.9539 

0.8188 

0.9524 

0.8826 

0.9134 

0.8188 

0.7828 

0.8826 

0TS8 

-0.07728 

-0.07463 

-0.11102 

-0.11147 

0.52530 

0.55539 

0.43636 

0.45005 

-0.08507 

0.3658 

0.3826 

0.1932 

0.1914 

<.0001 

<.0001 

<.0001 

<.0001 

0.3194 

ITS1 1 

0.62549 

0.63502 

-0.04448 

-0.03320 

0.76888 

0.76173 

-0.04448 

-0.03824 

-0.02869 

<.0001 

<.0001 

0.6031 

0.6980 

<.0001 

<.0001 

0.6031 

0.6549 

0.7374 

OTS11 

0.61868 

0.63026 

-0.02466 

-0.01232 

0.76965 

0.76631 

-0.02596 

-0.02017 

-0.04021 

<.0001 

<.0001 

0.7733 

0.8856 

<.0001 

<.0001 

0.7616 

0.8137 

0.6384 

ITS12 

-0.06068 

-0.07177 

0.21243 

0.19779 

-0.07459 

-0.08331 

-0.11566 

-0.11684 

-0.07459 

0.4779 

0.4011 

0.0120 

0.0196 

0.3828 

0.3295 

0.1751 

0.1707 

0.3828 

OTS12 

-0.05752 

-0.06926 

0.20415 

0.18986 

-0.07961 

-0.08776 

-0.11117 

-0.11268 

-0.07148 

0.5012 

0.4178 

0.0159 

0.0252 

0.3515 

0.3043 

0.1926 

0.1866 

0.4031 

ITS15 

-0.02080 

-0.02289 

0.15714 

0 . 1 6446 

-0.02557 

0.00332 

-0.03964 

-0.04043 

-0.02557 

0.8080 

0.7891 

0.0647 

0.0530 

0.7651 

0.9690 

0.6431 

0.6365 

0.7651 

OTS15 

-0.02198 

-0.02193 

0.16544 

0 . 1 7343 

0.00058 

0.03173 

-0.01676 

-0.01764 

-0.02951 

0.7973 

0.7978 

0.0516 

0.0412 

0.9946 

0.7108 

0.8448 

0.8367 

0.7302 

ITS18 

-0.04044 

-0.05426 

-0.07707 

-0.06562 

-0.04971 

-0.02212 

-0.07707 

-0.08097 

0.11479 

0.6365 

0.5258 

0.3672 

0.4428 

0.5612 

0.7961 

0.3672 

0.3434 

0.1784 

OTS18 

-0.04262 

-0.05660 

-0.07472 

-0.06308 

-0.04689 

-0.01795 

-0.06684 

-0.07059 

0.11267 

0.6184 

0.5081 

0.3820 

0.4607 

0.5836 

0.8339 

0.4343 

0.4089 

0.1867 

ITS19 

-0.03714 

-0.05111 

-0.07079 

-0.07354 

-0.04565 

-0.04775 

0.04635 

0.04467 

-0.04565 

0.6642 

0.5502 

0.4076 

0.3896 

0.5936 

0.5767 

0.5880 

0.6016 

0.5936 

OTS19 

-0.04209 

-0.05620 

-0.07022 

-0.07357 

-0.03374 

-0.03713 

0.03600 

0.03521 

-0.03983 

0.6228 

0.5111 

0.4114 

0.3894 

0.6934 

0 . 6644 

0.6740 

0.6807 

0.6416 

ITS20 

-0.01460 

-0.02131 

-0.02782 

-0.01249 

-0.01795 

-0.00811 

-0.02782 

-0.01920 

-0.01795 

0.8646 

0.8033 

0.7451 

0.8840 

0.8339 

0.9245 

0.7451 

0.8225 

0.8339 

0TS20 

-0.02563 

-0.03182 

-0.00983 

0.00632 

-0.03243 

-0.02276 

-0.03226 

-0.02373 

0.00410 

0.7645 

0.7100 

0.9086 

0.9411 

0.7047 

0.7903 

0.7062 

0.7816 

0.9618 

ITS22 

-0.01460 

-0.00924 

-0.02782 

-0.02415 

-0.01795 

0.00103 

-0.02782 

-0.03086 

-0.01795 

0.8646 

0.9140 

0.7451 

0.7778 

0.8339 

0.9904 

0.7451 

0.7184 

0.8339 
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Observed  to  Trained  NNS  Output  for  Exercises  1,2,3  28 

07:43  Wednesday,  January  15,  2003 

The  CORR  Procedure 

Pearson  Correlation  Coefficients,  N  =  139 
Prob  >  | r |  under  HO:  Rho=0 


0TA19 

ITA21 

0TA21 

ITA23 

OTA23 

I M 1 

0M1 

IM2 

0M2 

ITS8 

-0.01974 

-0.03691 

-0.03925 

0.17338 

0.16781 

-0.04331 

-0.04192 

0.15459 

0.14811 

0.8176 

0.6662 

0.6464 

0.0412 

0.0483 

0.6127 

0.6241 

0.0692 

0.0819 

0TS8 

-0.09422 

-0.25958 

-0.26642 

-0.00783 

0.00156 

0.19804 

0.19974 

-0.08978 

-0.08760 

0.2699 

0.0020 

0.0015 

0.9271 

0.9854 

0.0194 

0.0184 

0.2932 

0.3051 

ITS1 1 

-0.04450 

-0.08376 

-0.08885 

-0.09484 

-0.08708 

-0.09827 

-0.09356 

-0.03855 

-0.02424 

0.6029 

0.3269 

0.2983 

0.2667 

0.3081 

0.2498 

0.2733 

0.6523 

0.7769 

OTS11 

-0.05741 

-0.10914 

-0.11484 

-0.07786 

-0.06959 

-0.09488 

-0.09042 

-0.05457 

-0.03994 

0.5021 

0.2009 

0.1783 

0.3623 

0.4156 

0.2666 

0.2898 

0.5234 

0.6407 

ITS12 

-0.07868 

0.71595 

0.71767 

-0.01990 

-0.00994 

-0.14710 

-0.16070 

0.16319 

0.16545 

0.3572 

<.0001 

<.0001 

0.8162 

0.9075 

0.0840 

0.0588 

0.0549 

0.0516 

OTS12 

-0.07536 

0.72609 

0.72782 

-0.01733 

-0.00742 

-0.15012 

-0.16354 

0.17068 

0.17297 

0.3779 

<.0001 

<.0001 

0.8395 

0.9309 

0.0777 

0 . 0544 

0.0445 

0.0417 

ITS15 

-0.03202 

-0.07464 

-0.08035 

0.24181 

0.23897 

-0.08757 

-0.08653 

0.12674 

0.12506 

0.7083 

0.3825 

0.3471 

0.0041 

0.0046 

0.3053 

0.3111 

0.1371 

0.1424 

OTS15 

-0.03778 

-0.08816 

-0.09422 

0.27193 

0.27043 

-0.08905 

-0.08796 

0.11546 

0.11493 

0.6588 

0.3020 

0.2699 

0.0012 

0.0013 

0.2972 

0.3032 

0.1759 

0.1779 

ITS18 

0.10198 

-0.14512 

-0.14798 

0.56076 

0.57329 

-0.13414 

-0.13664 

-0.20674 

-0.20978 

0.2322 

0.0883 

0.0821 

<.0001 

<.0001 

0.1154 

0.1087 

0.0146 

0.0132 

OTS18 

0.09965 

-0.15281 

-0.15571 

0.57345 

0.58613 

-0.13623 

-0.13878 

-0.21099 

-0.21426 

0.2431 

0.0725 

0.0672 

<.0001 

<.0001 

0.1098 

0.1032 

0.0127 

0.0113 

ITS19 

-0.02880 

0.07725 

0.07918 

-0.15093 

-0.15422 

-0.08671 

-0.07911 

-0.17936 

-0.18617 

0.7365 

0.3661 

0.3541 

0.0761 

0.0699 

0.3101 

0.3546 

0.0346 

0.0282 

OTS19 

-0.02224 

0.08991 

0.09189 

-0.16048 

-0.16382 

-0.07948 

-0.07172 

-0.17727 

-0.18527 

0.7949 

0.2925 

0.2820 

0.0591 

0.0540 

0.3524 

0.4015 

0.0368 

0.0290 

ITS20 

-0.00497 

-0.05239 

-0.05399 

-0.05932 

-0.05678 

0.26715 

0.28785 

-0.12848 

-0.12233 

0.9537 

0.5402 

0.5279 

0.4879 

0.5067 

0.0015 

0.0006 

0.1317 

0.1514 

0TS20 

0.01805 

-0.01817 

-0.02012 

-0.04862 

-0.04584 

0.29435 

0.31437 

-0.14223 

-0.13618 

0.8330 

0.8319 

0.8142 

0.5698 

0.5921 

0.0004 

0.0002 

0.0949 

0.1099 

ITS22 

-0.02402 

-0.05239 

-0.05485 

0.24608 

0.24152 

-0.06147 

-0.05420 

0.04547 

0.05942 

0.7789 

0.5402 

0.5213 

0.0035 

0.0042 

0.4722 

0.5262 

0.5951 

0.4872 
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Observed  to  Trained  NNS  Output  for  Exercises  1,2,3 

07:43  Wednesday,  January  15, 

The  CORR  Procedure 

Pearson  Correlation  Coefficients,  N  =  139 
Prob  >  | r |  under  HO:  Rho=0 


I  M3 

0M3 

IM4 

0M4 

IM5 

0M5 

ITS8 

-0.03222 

-0.03254 

-0.01029 

0.00823 

-0.02795 

-0.01380 

0.7065 

0.7037 

0.9043 

0.9234 

0.7440 

0.8719 

0TS8 

-0.00295 

-0.00652 

-0.07728 

-0.02725 

-0.09180 

-0.11183 

0.9725 

0.9392 

0.3658 

0.7501 

0.2824 

0.1900 

ITS11 

-0.07311 

-0.06536 

0.62549 

0.62542 

-0.06342 

-0.05967 

0.3924 

0.4446 

<.0001 

<.0001 

0.4582 

0.4853 

OTS11 

-0.05124 

-0.04234 

0.61868 

0.62405 

-0.06073 

-0.05784 

0.5492 

0.6207 

<.0001 

<.0001 

0.4776 

0.4988 

ITS12 

-0.13363 

-0.13356 

-0.06068 

-0.11150 

-0.04796 

-0.03101 

0.1168 

0.1170 

0.4779 

0.1913 

0.5751 

0.7170 

OTS12 

-0.13463 

-0.13426 

-0.05752 

-0.10965 

-0.05295 

-0.03528 

0.1141 

0.1151 

0.5012 

0.1988 

0.5359 

0.6801 

ITS15 

0.38956 

0.39136 

-0.02080 

0.06679 

-0.05652 

-0.05342 

<.0001 

<.0001 

0.8080 

0.4347 

0.5087 

0.5322 

OTS15 

0.42516 

0.42722 

-0.02198 

0.06767 

-0.04620 

-0.04271 

<.0001 

<.0001 

0.7973 

0.4287 

0.5891 

0.6176 

ITS18 

0.45376 

0.46258 

-0.04044 

-0.07680 

0.42447 

0.42530 

<.0001 

<.0001 

0.6365 

0.3688 

<.0001 

<.0001 

OTS18 

0.46271 

0.47196 

-0.04262 

-0.07809 

0.42462 

0.42580 

<.0001 

<.0001 

0.6184 

0.3609 

<■0001 

<.0001 

ITS19 

-0.11634 

-0.12291 

-0.03714 

-0.09561 

-0.10093 

-0.11644 

0.1726 

0.1494 

0.6642 

0.2629 

0.2371 

0.1722 

OTS19 

-0.13068 

-0.13724 

-0.04209 

-0.10290 

-0.11262 

-0.12831 

0.1252 

0.1072 

0.6228 

0.2280 

0.1868 

0.1322 

ITS20 

-0.04573 

-0.03734 

-0.01460 

-0.03899 

-0.03967 

-0.02546 

0.5930 

0.6625 

0.8646 

0.6486 

0.6429 

0.7661 

0TS20 

-0.05556 

-0.04474 

-0.02563 

-0.05475 

-0.03398 

-0.01561 

0.5159 

0.6010 

0.7645 

0.5221 

0.6913 

0.8553 

ITS22 

0.14442 

0.15788 

-0.01460 

0.01028 

-0.03967 

-0.02516 

0.0899 

0.0634 

0.8646 

0.9044 

0.6429 

0.7687 

29 
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Observed  to  Trained  NNS  Output  for  Exercises  1,2,3  30 

07:43  Wednesday,  January  15,  2003 

The  CORR  Procedure 

Pearson  Correlation  Coefficients,  N  =  139 
Prob  >  | r |  under  HO:  Rho=0 


ITP2 

OTP2 

ITP3 

OTP3 

ITP4 

0TP4 

ITP5 

OTP5 

ITP7 

OTS22 

0.01021 

0.06024 

0.00420 

0.00266 

-0.00297 

0.03463 

0.31647 

0.32699 

0.01192 

0.9051 

0.4812 

0.9609 

0.9752 

0.9724 

0.6857 

0.0001 

<.0001 

0.8893 

ITS23 

-0.03283 

0.01972 

-0.08192 

-0.08618 

-0.03283 

-0.04891 

-0.08192 

-0.07091 

-0.09531 

0.7012 

0.8177 

0.3377 

0.3131 

0.7012 

0.5675 

0.3377 

0.4068 

0.2644 

OTS23 

-0.03579 

0.03119 

-0.07738 

-0.08184 

-0.03606 

-0.05231 

-0.07174 

-0.06086 

-0.08636 

0.6757 

0.7155 

0.3653 

0.3382 

0.6734 

0.5408 

0.4013 

0.4766 

0.3121 

ITA2 

-0.00725 

0.05090 

-0.01808 

-0.02195 

-0.00725 

-0.01463 

-0.01808 

-0.02147 

-0.02104 

0.9325 

0.5518 

0.8327 

0.7976 

0.9325 

0.8643 

0.8327 

0.8019 

0.8058 

OTA2 

-0.01618 

0.04540 

-0.04393 

-0.04917 

-0.01725 

-0.02954 

-0.04571 

-0.05248 

-0.01639 

0.8500 

0.5957 

0.6076 

0.5654 

0 . 8403 

0.7300 

0.5932 

0.5395 

0.8481 

ITA3 

-0.02617 

0.33787 

-0.06529 

-0.07331 

-0.02617 

-0.03342 

-0.06529 

-0.05000 

0.58396 

0.7598 

<.0001 

0.4451 

0.3911 

0.7598 

0.6961 

0.4451 

0.5589 

<.0001 

OTA3 

-0.02933 

0.34112 

-0.07082 

-0.07931 

-0.02395 

-0.02824 

-0.05793 

-0.04232 

0.59002 

0.7318 

<.0001 

0.4074 

0.3533 

0.7796 

0.7414 

0.4982 

0.6208 

<.0001 

ITA7 

-0.01808 

0.09990 

-0.0451 1 

-0.04855 

-0.01808 

-0.00780 

-0.0451 1 

-0.04640 

-0.05249 

0.8327 

0.2420 

0.5980 

0.5703 

0.8327 

0.9274 

0.5980 

0.5876 

0.5394 

0TA7 

-0.0041 1 

0.08701 

-0.04699 

-0.05102 

-0.01422 

-0.00181 

-0.04115 

-0.04266 

-0.04319 

0.9617 

0.3084 

0.5828 

0.5508 

0.8680 

0.9832 

0.6306 

0.6181 

0.6136 

ITA10 

-0.01029 

0 . 1  6474 

-0.02566 

-0.03115 

-0.01029 

0.01537 

-0.02566 

-0.02420 

-0.02986 

0.9043 

0.0526 

0.7643 

0.7158 

0.9043 

0.8574 

0.7643 

0.7774 

0.7271 

0TA10 

-0.01502 

0.19269 

-0.02056 

-0.02677 

0.00273 

0.03010 

-0.0021 1 

0.00003 

-0.01381 

0.8607 

0.0231 

0.8102 

0.7544 

0.9746 

0.7250 

0.9804 

0.9997 

0.8718 

ITA1 1 

-0.01960 

0.09092 

-0.04891 

-0.05513 

-0.01960 

0.00197 

0.11296 

0.12478 

-0.05691 

0.8188 

0.2871 

0.5675 

0.5192 

0.8188 

0.9816 

0.1855 

0.1433 

0.5058 

OTA1 1 

-0.02153 

0.11518 

-0.05099 

-0.05682 

-0.00552 

0.01766 

0.11977 

0.13312 

-0.05550 

0.8014 

0.1770 

0.5511 

0.5064 

0.9486 

0.8365 

0.1602 

0.1182 

0.5164 

ITA12 

-0.01264 

0.43531 

0.69926 

0.70014 

-0.01264 

-0.01697 

0.69926 

0.70071 

-0.03670 

0.8826 

<.0001 

<.0001 

<.0001 

0.8826 

0.8428 

<.0001 

<.0001 

0.6680 

OTA12 

0.00501 

0.45616 

0.71382 

0.71661 

-0.01644 

-0.01798 

0.71229 

0.71754 

-0.03950 

0.9533 

<.0001 

<.0001 

<.0001 

0.8477 

0.8336 

<.0001 

<.0001 

0.6443 

360 


Observed  to  Trained  NNS  Output  for  Exercises  1,2,3  31 

07:43  Wednesday,  January  15,  2003 

The  CORR  Procedure 

Pearson  Correlation  Coefficients,  N  =  139 
Prob  >  | r |  under  HO:  Rho=0 


0TP7 

ITP8 

0TP8 

ITP10 

0TP10 

ITP1 1 

0TP1 1 

ITP12 

0TP12 

OTS22 

0.03107 

0.01669 

0.04193 

0.16582 

0.16595 

0.02372 

0.02635 

-0.01724 

-0.01151 

0.7166 

0.8454 

0.6241 

0.0511 

0.0509 

0.7817 

0.7582 

0.8404 

0.8930 

ITS23 

-0.08008 

-0.04660 

-0.06274 

-0.13415 

-0.13849 

0.18300 

0.17944 

0.33921 

0.34546 

0.3487 

0.5859 

0.4631 

0.1154 

0.1040 

0.0311 

0.0345 

<.0001 

<.0001 

OTS23 

-0.07178 

-0.04100 

-0.05763 

-0.14067 

-0.14485 

0.17076 

0.16700 

0.34278 

0.34868 

0.4011 

0.6318 

0.5004 

0.0986 

0.0889 

0.0444 

0.0494 

<.0001 

<.0001 

ITA2 

-0.02387 

-0.01029 

-0.01675 

-0.02961 

-0.03151 

-0.07419 

-0.07788 

-0.02617 

-0.03026 

0.7803 

0 . 9043 

0.8448 

0.7294 

0.7127 

0.3854 

0.3622 

0.7598 

0.7236 

OTA2 

-0.02068 

-0.02146 

-0.02881 

-0.05304 

-0.05410 

-0.11160 

-0.11595 

-0.02536 

-0.03125 

0.8091 

0.8020 

0.7364 

0.5352 

0.5271 

0.1909 

0.1740 

0.7669 

0.7149 

ITA3 

0.59610 

-0.03714 

-0.04060 

-0.10691 

-0.11148 

0.04242 

0 . 04043 

-0.09449 

-0.08972 

<.0001 

0.6642 

0.6351 

0.2103 

0.1914 

0.6201 

0.6365 

0.2685 

0.2935 

OTA3 

0.60279 

-0.02682 

-0.03138 

-0.11252 

-0.11676 

0.04471 

0.04249 

-0.09097 

-0.08642 

<.0001 

0.7540 

0.7139 

0.1872 

0.1711 

0.6012 

0.6195 

0.2868 

0.3118 

ITA7 

-0.04373 

-0.02566 

-0.01048 

-0.07387 

-0.07932 

-0.18510 

-0.19044 

-0.06529 

-0.07497 

0.6092 

0.7643 

0.9026 

0.3874 

0.3533 

0.0291 

0.0247 

0.4451 

0.3804 

0TA7 

-0.03458 

-0.01301 

0.00141 

-0.09197 

-0.09796 

-0.19289 

-0.19848 

-0.08189 

-0.09246 

0.6861 

0.8792 

0.9868 

0.2815 

0.2513 

0.0229 

0.0192 

0.3379 

0.2790 

ITA10 

-0.02740 

-0.01460 

-0.02378 

-0.04202 

-0.03750 

-0.10530 

-0.11028 

-0.03714 

-0.04205 

0.7489 

0.8646 

0.7812 

0.6233 

0.6612 

0.2173 

0.1962 

0.6642 

0.6231 

0TA10 

-0.01093 

-0.02079 

-0.03040 

-0.05128 

-0.04649 

-0.12774 

-0.13301 

-0.04176 

-0.04712 

0.8984 

0.8081 

0.7224 

0.5488 

0.5868 

0.1340 

0.1185 

0.6254 

0.5818 

ITA1 1 

-0.06487 

-0.02782 

-0.04413 

-0.08009 

-0.07431 

-0.20069 

-0.20244 

-0.07079 

-0.07537 

0.4481 

0.7451 

0.6060 

0.3486 

0.3846 

0.0178 

0.0168 

0.4076 

0.3779 

OTA1 1 

-0.06281 

-0.03376 

-0.05067 

-0.06504 

-0.05941 

-0.20528 

-0.20742 

-0.07181 

-0.07691 

0.4626 

0.6931 

0.5536 

0.4469 

0.4873 

0.0153 

0.0143 

0.4009 

0.3682 

ITA12 

-0.04297 

-0.01795 

-0.03012 

-0.05166 

-0.03591 

-0.12944 

-0.12307 

-0.04565 

-0.03640 

0.6154 

0.8339 

0.7249 

0.5459 

0.6747 

0.1289 

0.1489 

0.5936 

0.6706 

OTA12 

-0.04421 

-0.01507 

-0.02484 

-0.01141 

0.00438 

-0.09835 

-0.09206 

-0.01934 

-0.00898 

0.6053 

0.8602 

0.7716 

0.8940 

0.9592 

0.2494 

0.2811 

0.8212 

0.9164 
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Observed  to  Trained  NNS  Output  for  Exercises  1,2,3  32 

07:43  Wednesday,  January  15,  2003 

The  CORR  Procedure 

Pearson  Correlation  Coefficients,  N  =  139 
Prob  >  | r |  under  HO:  Rho=0 


ITP15 

0TP15 

ITP22 

0TP22 

ITP23 

0TP23 

ITS1 

OTS1 

ITS2 

OTS22 

-0.03278 

-0.02265 

-0.01371 

-0.00881 

-0.03196 

-0.02866 

-0.01304 

-0.01535 

0.00388 

0.7016 

0.7913 

0.8727 

0.9180 

0.7088 

0.7377 

0.8789 

0.8576 

0.9639 

ITS23 

-0.12389 

-0.12468 

0.49908 

0.49188 

-0.13415 

-0.12856 

-0.03283 

-0.02908 

-0.08192 

0.1462 

0.1436 

<■0001 

<.0001 

0 . 1 1 54 

0.1315 

0.7012 

0.7340 

0.3377 

OTS23 

-0.12604 

-0.12669 

0.50055 

0.49454 

-0.13329 

-0.12765 

-0.02277 

-0.01834 

-0.07494 

0.1393 

0.1372 

<.0001 

<.0001 

0.1178 

0.1342 

0.7902 

0.8303 

0.3806 

ITA2 

-0.02734 

-0.01825 

0.36966 

0.37610 

-0.02961 

-0.02703 

-0.00725 

-0.00570 

-0.01808 

0.7493 

0.8312 

<.0001 

<.0001 

0.7294 

0.7521 

0.9325 

0.9469 

0.8327 

OTA2 

-0.03288 

-0.02235 

0.42595 

0.43439 

0.01468 

0.01738 

0.00196 

0.00272 

-0.00621 

0.7008 

0.7940 

<.0001 

<.0001 

0.8638 

0.8391 

0.9818 

0.9746 

0.9422 

ITA3 

-0.09874 

-0.09814 

-0.07079 

-0.07230 

-0.10691 

-0.10052 

-0.02617 

-0.04078 

0.69097 

0.2475 

0.2504 

0.4076 

0.3977 

0.2103 

0.2390 

0.7598 

0.6336 

<.0001 

OTA3 

-0.09112 

-0.09047 

-0.05575 

-0.05731 

-0.11028 

-0.10354 

-0.02933 

-0.04353 

0.69187 

0.2861 

0.2895 

0.5145 

0.5028 

0.1962 

0.2251 

0.7318 

0.6109 

<.0001 

ITA7 

-0.06822 

-0.06933 

0.59858 

0.61256 

0.15431 

0.15760 

0.40078 

0.46189 

-0.0451 1 

0.4249 

0.4173 

<.0001 

<.0001 

0.0697 

0.0639 

<.0001 

<.0001 

0.5980 

0TA7 

-0.08325 

-0.08471 

0.61349 

0.62827 

0.14838 

0.15186 

0.39446 

0.45602 

-0.04188 

0.3299 

0.3214 

<.0001 

<.0001 

0.0813 

0 . 0743 

<.0001 

<.0001 

0.6245 

ITA10 

0.37616 

0.37837 

-0.02782 

-0.01881 

-0.04202 

-0.04484 

-0.01029 

0.00652 

-0.02566 

<.0001 

<.0001 

0.7451 

0.8260 

0.6233 

0.6002 

0.9043 

0.9393 

0 . 7643 

0TA10 

0.38135 

0.38461 

-0.00748 

0.00186 

-0.05350 

-0.05584 

-0.00466 

0.01316 

-0.00365 

<.0001 

<.0001 

0.9304 

0.9827 

0.5317 

0.5138 

0.9565 

0.8778 

0.9660 

ITA1 1 

0.15200 

0.15796 

-0.05303 

-0.05014 

0.34402 

0.33899 

-0.01960 

-0.01290 

-0.04891 

0.0741 

0.0633 

0.5353 

0.5578 

<.0001 

<.0001 

0.8188 

0.8802 

0.5675 

OTA1 1 

0.17072 

0.17756 

-0.03666 

-0.03324 

0.35003 

0.34594 

-0.01907 

-0.01023 

-0.04381 

0.0445 

0.0365 

0.6683 

0.6977 

<.0001 

<.0001 

0.8237 

0.9048 

0.6086 

ITA12 

-0.04771 

-0.03325 

-0.03420 

-0.03758 

-0.05166 

-0.03804 

-0.01264 

-0.00112 

-0.03155 

0.5771 

0.6975 

0.6894 

0.6605 

0.5459 

0.6566 

0.8826 

0.9896 

0.7124 

OTA12 

-0.03482 

-0.01963 

-0.04448 

-0.04833 

-0.04447 

-0.02990 

-0.01522 

-0.00274 

-0.03643 

0.6841 

0.8186 

0.6031 

0.5721 

0.6032 

0.7268 

0.8589 

0.9744 

0.6702 
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Observed  to  Trained  NNS  Output  for  Exercises  1,2,3  33 
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The  CORR  Procedure 

Pearson  Correlation  Coefficients,  N  =  139 
Prob  >  | r |  under  HO:  Rho=0 


OTS2 

ITS5 

OTS5 

ITS8 

0TS8 

ITS1 1 

0TS11 

ITS12 

0TS12 

OTS22 

0.01415 

-0.02674 

-0.01821 

0.03811 

-0.05492 

-0.00181 

0.01043 

-0.06502 

-0.05922 

0.8687 

0.7547 

0.8315 

0.6561 

0.5208 

0.9832 

0.9030 

0.4470 

0.4887 

ITS23 

-0.06497 

-0.08192 

-0.08641 

-0.03283 

0.42770 

-0.07450 

-0.06145 

-0.19371 

-0.19501 

0.4473 

0.3377 

0.3118 

0.7012 

<.0001 

0.3834 

0.4724 

0.0223 

0.0214 

OTS23 

-0.05800 

-0.07925 

-0.08389 

-0.03579 

0.43397 

-0.05831 

-0.04527 

-0.20340 

-0.20512 

0.4976 

0.3537 

0.3262 

0.6757 

<.0001 

0.4953 

0.5967 

0.0163 

0.0154 

ITA2 

-0.00002 

-0.01808 

-0.02174 

-0.00725 

-0.06298 

-0.01644 

-0.01789 

-0.04275 

-0.04452 

0.9999 

0.8327 

0.7995 

0.9325 

0.4614 

0.8476 

0.8345 

0.6173 

0.6028 

OTA2 

0.01335 

-0.04038 

-0.04515 

-0.01832 

-0.10011 

-0.01880 

-0.02060 

-0.05816 

-0.06023 

0.8760 

0.6370 

0.5977 

0.8305 

0.2410 

0.8261 

0.8098 

0.4965 

0.4812 

ITA3 

0.69667 

-0.06529 

-0.06484 

-0.02617 

0.17312 

-0.05938 

-0.06051 

-0.15439 

-0.15956 

<.0001 

0.4451 

0.4482 

0.7598 

0.0415 

0.4875 

0.4792 

0.0696 

0.0606 

OTA3 

0.69838 

-0.04859 

-0.04782 

-0.01605 

0.17259 

-0.05019 

-0.05130 

-0.15233 

-0.15777 

<.0001 

0.5700 

0.5762 

0.8513 

0.0422 

0.5573 

0.5487 

0.0734 

0.0636 

ITA7 

-0.03253 

-0.0451 1 

-0.04013 

-0.01808 

-0.11457 

-0.04103 

-0.04443 

-0.10668 

-0.10563 

0.7038 

0.5980 

0.6391 

0.8327 

0.1793 

0.6315 

0.6035 

0.2113 

0.2159 

0TA7 

-0.02874 

-0.03493 

-0.02968 

-0.00104 

-0.13544 

-0.05121 

-0.05600 

-0.10307 

-0.10205 

0.7370 

0.6831 

0.7287 

0.9903 

0.1119 

0.5493 

0.5126 

0.2273 

0.2319 

ITA10 

-0.02590 

-0.02566 

-0.01302 

-0.01029 

-0.07728 

0.62549 

0.61868 

-0.06068 

-0.05752 

0.7622 

0 . 7643 

0.8790 

0 . 9043 

0.3658 

<.0001 

<.0001 

0.4779 

0.5012 

0TA10 

-0.00279 

-0.00611 

0.00646 

0.00495 

-0.07463 

0.63502 

0.63026 

-0.07177 

-0.06926 

0.9740 

0.9431 

0.9398 

0.9539 

0.3826 

<.0001 

<.0001 

0.4011 

0.4178 

ITA1 1 

-0.04232 

0.27484 

0.27032 

-0.01960 

-0.11102 

-0.04448 

-0.02466 

0.21243 

0.20415 

0.6209 

0.0011 

0.0013 

0.8188 

0.1932 

0.6031 

0.7733 

0.0120 

0.0159 

OTA1 1 

-0.03613 

0.27635 

0.27271 

-0.00510 

-0.11147 

-0.03320 

-0.01232 

0.19779 

0.18986 

0.6728 

0.0010 

0.0012 

0.9524 

0.1914 

0.6980 

0.8856 

0.0196 

0.0252 

ITA12 

-0.03537 

-0.03155 

-0.03767 

-0.01264 

0.52530 

0.76888 

0.76965 

-0.07459 

-0.07961 

0.6793 

0.7124 

0.6597 

0.8826 

<.0001 

<.0001 

<.0001 

0.3828 

0.3515 

OTA12 

-0.03972 

-0.04128 

-0.04768 

0.00931 

0.55539 

0.76173 

0.76631 

-0.08331 

-0.08776 

0.6425 

0.6295 

0.5773 

0.9134 

<.0001 

<.0001 

<.0001 

0.3295 

0.3043 
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Observed  to  Trained  NNS  Output  for  Exercises  1,2,3  34 
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The  CORR  Procedure 

Pearson  Correlation  Coefficients,  N  =  139 
Prob  >  | r |  under  HO:  Rho=0 


ITS15 

0TS15 

ITS18 

0TS18 

ITS19 

0TS19 

ITS20 

0TS20 

ITS22 

OTS22 

-0.01305 

-0.00437 

-0.01166 

-0.00009 

-0.02750 

-0.03144 

-0.02346 

0.00009 

0.99121 

0.8788 

0.9592 

0.8916 

0.9991 

0.7479 

0.7133 

0.7840 

0.9992 

<.0001 

ITS23 

-0.06639 

-0.05206 

-0.12908 

-0.12704 

-0.11856 

-0.11909 

-0.04660 

-0.06781 

-0.04660 

0.4374 

0.5428 

0.1299 

0.1361 

0.1645 

0.1626 

0.5859 

0.4276 

0.5859 

OTS23 

-0.06351 

-0.04955 

-0.13430 

-0.13237 

-0.12326 

-0.12387 

-0.04741 

-0.06761 

-0.04496 

0.4576 

0.5624 

0.1150 

0.1203 

0.1483 

0.1463 

0.5794 

0.4291 

0.5992 

ITA2 

-0.01465 

-0.01869 

-0.02849 

-0.02943 

-0.02617 

-0.01394 

-0.01029 

-0.01011 

-0.01029 

0.8641 

0.8271 

0.7392 

0.7309 

0.7598 

0.8706 

0.9043 

0.9060 

0.9043 

OTA2 

-0.03704 

-0.04477 

-0.03216 

-0.03288 

0.02280 

0.03688 

-0.02146 

-0.02008 

-0.02373 

0.6651 

0.6007 

0.7070 

0.7008 

0.7899 

0.6665 

0.8020 

0.8145 

0.7816 

ITA3 

-0.05291 

-0.05513 

-0.10287 

-0.10797 

-0.09449 

-0.10144 

-0.03714 

-0.03506 

-0.03714 

0.5362 

0.5192 

0.2282 

0.2058 

0.2685 

0.2348 

0.6642 

0.6820 

0.6642 

OTA3 

-0.05116 

-0.05256 

-0.11069 

-0.11585 

-0.09763 

-0.10435 

-0.04163 

-0.03838 

-0.03580 

0.5498 

0.5389 

0.1945 

0.1745 

0.2529 

0.2215 

0.6265 

0.6537 

0.6757 

ITA7 

-0.03656 

-0.04574 

-0.07108 

-0.07406 

-0.06529 

-0.05524 

0.56886 

0.59543 

-0.02566 

0.6692 

0.5928 

0.4057 

0.3863 

0.4451 

0.5183 

<.0001 

<.0001 

0.7643 

0TA7 

-0.04608 

-0.05616 

-0.08219 

-0.08588 

-0.06946 

-0.05877 

0.56923 

0.59775 

-0.01956 

0.5901 

0.5114 

0.3361 

0.3148 

0.4165 

0.4920 

<.0001 

<.0001 

0.8192 

ITA10 

-0.02080 

-0.02198 

-0.04044 

-0.04262 

-0.03714 

-0.04209 

-0.01460 

-0.02563 

-0.01460 

0.8080 

0.7973 

0.6365 

0.6184 

0.6642 

0.6228 

0.8646 

0.7645 

0.8646 

0TA10 

-0.02289 

-0.02193 

-0.05426 

-0.05660 

-0.05111 

-0.05620 

-0.02131 

-0.03182 

-0.00924 

0.7891 

0.7978 

0.5258 

0.5081 

0.5502 

0.5111 

0.8033 

0.7100 

0.9140 

ITA1 1 

0.15714 

0.16544 

-0.07707 

-0.07472 

-0.07079 

-0.07022 

-0.02782 

-0.00983 

-0.02782 

0.0647 

0.0516 

0.3672 

0.3820 

0.4076 

0.4114 

0.7451 

0.9086 

0.7451 

OTA1 1 

0.16446 

0 . 1 7343 

-0.06562 

-0.06308 

-0.07354 

-0.07357 

-0.01249 

0.00632 

-0.02415 

0.0530 

0.0412 

0.4428 

0.4607 

0.3896 

0.3894 

0.8840 

0.9411 

0.7778 

ITA12 

-0.02557 

0.00058 

-0.04971 

-0.04689 

-0.04565 

-0.03374 

-0.01795 

-0.03243 

-0.01795 

0.7651 

0.9946 

0.5612 

0.5836 

0.5936 

0.6934 

0.8339 

0.7047 

0.8339 

OTA12 

0.00332 

0.03173 

-0.02212 

-0.01795 

-0.04775 

-0.03713 

-0.00811 

-0.02276 

0.00103 

0.9690 

0.7108 

0.7961 

0.8339 

0.5767 

0.6644 

0.9245 

0.7903 

0.9904 
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Observed  to  Trained  NNS  Output  for  Exercises  1,2,3  35 

07:43  Wednesday,  January  15,  2003 

The  CORR  Procedure 

Pearson  Correlation  Coefficients,  N  =  139 
Prob  >  | r |  under  HO:  Rho=0 


0TS22 

ITS23 

OTS23 

ITA2 

OTA2 

ITA3 

OTA3 

ITA7 

0TA7 

OTS22 

1  .00000 

-0.04036 

-0.03772 

0.01098 

-0.00020 

-0.03006 

-0.02709 

-0.02255 

-0.01582 

0.6371 

0.6593 

0.8979 

0.9981 

0.7253 

0.7515 

0.7922 

0.8533 

ITS23 

-0.04036 

1 .00000 

0.99863 

0.22071 

0.24176 

0.33921 

0.35416 

0.23438 

0.23622 

0.6371 

<.0001 

0.0090 

0.0041 

<.0001 

<.0001 

0.0055 

0.0051 

OTS23 

-0.03772 

0.99863 

1 .00000 

0.21666 

0.23839 

0.34230 

0.35765 

0.24022 

0.24236 

0.6593 

<.0001 

0.0104 

0.0047 

<.0001 

<.0001 

0.0044 

0.0040 

ITA2 

0.01098 

0.22071 

0.21666 

1 .00000 

0.98799 

-0.02617 

-0.01921 

-0.01808 

-0.00104 

0.8979 

0.0090 

0.0104 

<.0001 

0.7598 

0.8224 

0.8327 

0.9903 

OTA2 

-0.00020 

0.24176 

0.23839 

0.98799 

1 .00000 

-0.03403 

-0.02717 

0.02264 

0.03877 

0.9981 

0.0041 

0.0047 

<.0001 

0.6908 

0.7509 

0.7914 

0.6505 

ITA3 

-0.03006 

0.33921 

0.34230 

-0.02617 

-0.03403 

1 .00000 

0.99872 

-0.06529 

-0.06576 

0.7253 

<.0001 

<.0001 

0.7598 

0.6908 

<.0001 

0.4451 

0.4418 

OTA3 

-0.02709 

0.35416 

0.35765 

-0.01921 

-0.02717 

0.99872 

1 .00000 

-0.05937 

-0.05910 

0.7515 

<.0001 

<.0001 

0.8224 

0.7509 

<.0001 

0.4875 

0.4895 

ITA7 

-0.02255 

0.23438 

0.24022 

-0.01808 

0.02264 

-0.06529 

-0.05937 

1 .00000 

0.99755 

0.7922 

0.0055 

0.0044 

0.8327 

0.7914 

0.4451 

0.4875 

<.0001 

0TA7 

-0.01582 

0.23622 

0.24236 

-0.00104 

0.03877 

-0.06576 

-0.05910 

0.99755 

1 .00000 

0.8533 

0.0051 

0.0040 

0.9903 

0.6505 

0.4418 

0.4895 

<.0001 

ITA10 

-0.0141 1 

-0.04660 

-0.03572 

-0.01029 

0.00959 

-0.03714 

-0.02053 

-0.02566 

-0.03203 

0.8691 

0.5859 

0.6764 

0 . 9043 

0.9107 

0.6642 

0.8104 

0 . 7643 

0.7081 

0TA10 

-0.00501 

-0.03174 

-0.02017 

0.00421 

0.02449 

-0.02129 

-0.00352 

-0.00949 

-0.01565 

0.9533 

0.7107 

0.8136 

0.9608 

0.7747 

0.8035 

0.9672 

0.9117 

0.8549 

ITA1 1 

-0.02030 

-0.08882 

-0.08143 

-0.01960 

-0.02687 

-0.07079 

-0.06297 

-0.04891 

-0.05477 

0.8125 

0.2985 

0.3406 

0.8188 

0.7535 

0.4076 

0.4615 

0.5675 

0.5219 

OTA1 1 

-0.01497 

-0.07739 

-0.06998 

-0.00346 

-0.01136 

-0.05859 

-0.04992 

-0.04040 

-0.04534 

0.8611 

0.3652 

0.4130 

0.9677 

0.8944 

0.4933 

0.5595 

0.6368 

0.5961 

ITA12 

0.00925 

-0.05728 

-0.04545 

-0.01264 

-0.03196 

-0.04565 

-0.04750 

-0.03155 

-0.03938 

0.9140 

0.5030 

0.5952 

0.8826 

0.7088 

0.5936 

0.5787 

0.7124 

0.6453 

OTA12 

0.03204 

-0.04163 

-0.03024 

-0.01644 

-0.03933 

-0.03576 

-0.03741 

-0.03159 

-0.03997 

0.7080 

0.6266 

0.7238 

0.8477 

0.6457 

0.6760 

0.6620 

0.7120 

0.6403 
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Observed  to  Trained  NNS  Output  for  Exercises  1,2,3  36 
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The  CORR  Procedure 

Pearson  Correlation  Coefficients,  N  =  139 
Prob  >  | r |  under  HO:  Rho=0 


ITA10 

0TA10 

ITA1 1 

0TA1 1 

ITA12 

0TA12 

ITA18 

0TA18 

ITA19 

OTS22 

-0.01411 

-0.00501 

-0.02030 

-0.01497 

0.00925 

0.03204 

-0.01761 

-0.02030 

-0.00292 

0.8691 

0.9533 

0.8125 

0.8611 

0.9140 

0.7080 

0.8370 

0.8125 

0.9728 

ITS23 

-0.04660 

-0.03174 

-0.08882 

-0.07739 

-0.05728 

-0.04163 

0.49908 

0.52030 

0.23762 

0.5859 

0.7107 

0.2985 

0.3652 

0.5030 

0.6266 

<.0001 

<.0001 

0.0049 

OTS23 

-0.03572 

-0.02017 

-0.08143 

-0.06998 

-0.04545 

-0.03024 

0.50672 

0.52825 

0.23922 

0.6764 

0.8136 

0.3406 

0.4130 

0.5952 

0.7238 

<.0001 

<.0001 

0.0046 

ITA2 

-0.01029 

0.00421 

-0.01960 

-0.00346 

-0.01264 

-0.01644 

-0.01960 

-0.02153 

-0.01264 

0.9043 

0.9608 

0.8188 

0.9677 

0.8826 

0.8477 

0.8188 

0.8013 

0.8826 

OTA2 

0.00959 

0.02449 

-0.02687 

-0.01136 

-0.03196 

-0.03933 

-0.01450 

-0.01537 

0.03631 

0.9107 

0.7747 

0.7535 

0 . 8944 

0.7088 

0.6457 

0.8655 

0.8575 

0.6713 

ITA3 

-0.03714 

-0.02129 

-0.07079 

-0.05859 

-0.04565 

-0.03576 

-0.07079 

-0.06466 

-0.04565 

0.6642 

0.8035 

0.4076 

0.4933 

0.5936 

0.6760 

0.4076 

0.4495 

0.5936 

OTA3 

-0.02053 

-0.00352 

-0.06297 

-0.04992 

-0.04750 

-0.03741 

-0.06248 

-0.05634 

-0.04087 

0.8104 

0.9672 

0.4615 

0.5595 

0.5787 

0.6620 

0.4650 

0.5100 

0.6328 

ITA7 

-0.02566 

-0.00949 

-0.04891 

-0.04040 

-0.03155 

-0.03159 

-0.04891 

-0.03171 

-0.03155 

0.7643 

0.9117 

0.5675 

0.6368 

0.7124 

0.7120 

0.5675 

0.7110 

0.7124 

0TA7 

-0.03203 

-0.01565 

-0.05477 

-0.04534 

-0.03938 

-0.03997 

-0.05341 

-0.03697 

-0.01689 

0.7081 

0.8549 

0.5219 

0.5961 

0.6453 

0 . 6403 

0.5323 

0.6657 

0.8436 

ITA10 

1  .00000 

0.99663 

-0.02782 

-0.00637 

-0.01795 

-0.02464 

-0.02782 

-0.03260 

-0.01795 

<.0001 

0.7451 

0.9407 

0.8339 

0.7734 

0.7451 

0.7032 

0.8339 

0TA10 

0.99663 

1 .00000 

-0.00576 

0.01635 

-0.00298 

-0.00928 

-0.02491 

-0.02966 

-0.02706 

<.0001 

0 . 9463 

0.8485 

0.9722 

0.9137 

0.7710 

0.7289 

0.7519 

ITA1 1 

-0.02782 

-0.00576 

1 .00000 

0.99750 

-0.03420 

-0.03050 

-0.05303 

-0.04271 

-0.03420 

0.7451 

0.9463 

<.0001 

0.6894 

0.7215 

0.5353 

0.6177 

0.6894 

OTA1 1 

-0.00637 

0.01635 

0.99750 

1 .00000 

-0.03732 

-0.03194 

-0.05634 

-0.04568 

-0.01894 

0.9407 

0.8485 

<.0001 

0.6627 

0.7090 

0.5101 

0.5933 

0.8249 

ITA12 

-0.01795 

-0.00298 

-0.03420 

-0.03732 

1 .00000 

0.99632 

-0.03420 

-0.02229 

-0.02206 

0.8339 

0.9722 

0.6894 

0.6627 

<.0001 

0.6894 

0.7945 

0.7966 

OTA12 

-0.02464 

-0.00928 

-0.03050 

-0.03194 

0.99632 

1 .00000 

-0.01984 

-0.00800 

-0.02940 

0.7734 

0.9137 

0.7215 

0.7090 

<.0001 

0.8167 

0.9256 

0.7312 
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Observed  to  Trained  NNS  Output  for  Exercises  1,2,3  37 

07:43  Wednesday,  January  15,  2003 

The  CORR  Procedure 

Pearson  Correlation  Coefficients,  N  =  139 
Prob  >  | r |  under  HO:  Rho=0 


0TA19 

ITA21 

0TA21 

ITA23 

0TA23 

IM1 

0M1 

IM2 

0M2 

OTS22 

-0.00813 

-0.07594 

-0.07832 

0.27988 

0.27608 

-0.06453 

-0.05654 

-0.00031 

0.01418 

0.9244 

0.3743 

0.3594 

0.0008 

0.0010 

0.4504 

0.5085 

0.9971 

0.8684 

ITS23 

0.25435 

-0.16725 

-0.17626 

-0.18937 

-0.18469 

0.61320 

0.61940 

-0.41015 

-0.40504 

0.0025 

0.0491 

0.0379 

0.0256 

0.0295 

<.0001 

<.0001 

<.0001 

<.0001 

OTS23 

0.25667 

-0.18357 

-0.19281 

-0.19436 

-0.18975 

0.62502 

0.63081 

-0.42246 

-0.41752 

0.0023 

0.0305 

0.0230 

0.0219 

0.0253 

<.0001 

<.0001 

<.0001 

<.0001 

ITA2 

0.01149 

-0.03691 

-0.03925 

-0.04180 

-0.04437 

0.05959 

0.05976 

-0.09052 

-0.09559 

0.8932 

0.6662 

0 . 6464 

0.6252 

0.6040 

0.4859 

0.4847 

0.2892 

0.2630 

OTA2 

0.06429 

-0.04760 

-0.04962 

-0.08354 

-0.08621 

0.06940 

0.07086 

-0.13153 

-0.13913 

0.4521 

0.5779 

0.5619 

0.3282 

0.3129 

0.4169 

0.4072 

0.1227 

0.1024 

ITA3 

-0.04289 

-0.13329 

-0.13509 

-0.15093 

-0.14621 

0.44742 

0.44883 

-0.32688 

-0.33181 

0.6161 

0.1177 

0.1128 

0.0761 

0.0859 

<.0001 

<.0001 

<.0001 

<.0001 

OTA3 

-0.03814 

-0.13513 

-0.13724 

-0.14512 

-0.14050 

0.45989 

0.46130 

-0.34356 

-0.34817 

0.6558 

0.1127 

0.1072 

0.0883 

0.0990 

<.0001 

<.0001 

<.0001 

<.0001 

ITA7 

-0.02224 

-0.09210 

-0.09572 

-0.10429 

-0.10493 

0.46962 

0.48826 

-0.22586 

-0.22625 

0.7950 

0.2809 

0.2623 

0.2218 

0.2189 

<.0001 

<.0001 

0.0075 

0.0074 

0TA7 

-0.00727 

-0.08821 

-0.09182 

-0.10145 

-0.10298 

0.47616 

0.49508 

-0.21761 

-0.21899 

0.9323 

0.3018 

0.2824 

0.2347 

0.2277 

<.0001 

<.0001 

0.0101 

0.0096 

ITA10 

-0.02757 

-0.05239 

-0.05692 

-0.05932 

-0.06043 

-0.06147 

-0.06424 

-0.12848 

-0.11232 

0 . 7473 

0.5402 

0.5057 

0.4879 

0.4798 

0.4722 

0.4525 

0.1317 

0.1880 

0TA10 

-0.03742 

-0.07033 

-0.07495 

-0.06402 

-0.06495 

-0.05273 

-0.05537 

-0.14305 

-0.12593 

0.6618 

0.4107 

0.3806 

0.4540 

0.4475 

0.5376 

0.5174 

0.0930 

0.1396 

ITA1 1 

-0.02202 

-0.09986 

-0.10834 

-0.11307 

-0.09824 

0.04191 

0.03624 

-0.24488 

-0.23861 

0.7970 

0.2422 

0.2043 

0.1851 

0.2499 

0.6242 

0.6719 

0.0037 

0.0047 

OTA1 1 

-0.00761 

-0.12060 

-0.12980 

-0.09277 

-0.07801 

0.05897 

0.05383 

-0.26375 

-0.25819 

0.9292 

0.1573 

0.1278 

0.2774 

0.3614 

0.4905 

0.5291 

0.0017 

0.0021 

ITA12 

-0.03443 

-0.06440 

-0.06721 

-0.07292 

-0.06206 

-0.07556 

-0.06725 

0.05589 

0.06098 

0.6874 

0.4513 

0.4318 

0.3936 

0.4680 

0.3767 

0.4315 

0.5134 

0.4758 

OTA12 

-0.04370 

-0.08290 

-0.08644 

-0.02778 

-0.01655 

-0.06050 

-0.05219 

0.03783 

0.04333 

0.6094 

0.3319 

0.3116 

0.7454 

0.8467 

0.4793 

0.5417 

0.6584 

0.6125 
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Observed  to  Trained  NNS  Output  for  Exercises  1,2,3 

07:43  Wednesday,  January  15, 

The  CORR  Procedure 

Pearson  Correlation  Coefficients,  N  =  139 
Prob  >  | r |  under  HO:  Rho=0 


IM3 

0M3 

IM4 

0M4 

IM5 

0M5 

OTS22 

0.16219 

0.17703 

-0.01411 

0.00910 

0.00390 

0.01816 

0.0564 

0.0371 

0.8691 

0.9153 

0.9636 

0.8320 

ITS23 

-0.06408 

-0.06438 

-0.04660 

-0.05256 

-0.12664 

-0.13865 

0.4536 

0.4514 

0.5859 

0.5388 

0.1374 

0.1036 

OTS23 

-0.06662 

-0.06628 

-0.03572 

-0.04017 

-0.12625 

-0.13848 

0.4358 

0.4382 

0.6764 

0.6387 

0.1386 

0.1040 

ITA2 

-0.03222 

-0.03216 

-0.01029 

-0.02549 

-0.02795 

-0.02994 

0.7065 

0.7071 

0.9043 

0.7658 

0.7440 

0.7265 

OTA2 

-0.05865 

-0.05807 

0.00959 

-0.01645 

-0.02038 

-0.02262 

0.4928 

0.4971 

0.9107 

0.8476 

0.8118 

0.7915 

ITA3 

-0.11634 

-0.12420 

-0.03714 

0.00586 

0.32853 

0.33294 

0.1726 

0.1452 

0.6642 

0.9454 

<.0001 

<.0001 

OTA3 

-0.11014 

-0.11768 

-0.02053 

0.02242 

0.32882 

0.33406 

0.1968 

0.1677 

0.8104 

0.7934 

<.0001 

<.0001 

ITA7 

-0.08039 

-0.06305 

-0.02566 

-0.05864 

-0.06974 

-0.05800 

0.3469 

0.4609 

0.7643 

0.4929 

0.4146 

0.4976 

0TA7 

-0.09234 

-0.07481 

-0.03203 

-0.06591 

-0.07437 

-0.06258 

0.2796 

0.3814 

0.7081 

0.4407 

0.3842 

0.4643 

ITA10 

-0.04573 

-0.03790 

1  .00000 

0.96609 

-0.03967 

-0.02458 

0.5930 

0.6578 

<.0001 

<.0001 

0.6429 

0.7740 

0TA10 

-0.03407 

-0.02596 

0.99663 

0.96785 

-0.03161 

-0.01580 

0.6905 

0.7616 

<.0001 

<.0001 

0.7118 

0.8536 

ITA1 1 

0.29004 

0.30151 

-0.02782 

0.06673 

-0.07561 

-0.07634 

0.0005 

0.0003 

0.7451 

0.4351 

0.3763 

0.3717 

OTA1 1 

0.30219 

0.31496 

-0.00637 

0.09081 

-0.06615 

-0.06532 

0.0003 

0.0002 

0.9407 

0.2877 

0.4391 

0.4448 

ITA12 

-0.05621 

-0.05270 

-0.01795 

0.00975 

-0.04877 

-0.05632 

0.5110 

0.5378 

0.8339 

0.9093 

0.5686 

0.5102 

OTA12 

-0.02649 

-0.02210 

-0.02464 

0.00623 

-0.04551 

-0.05207 

0.7569 

0.7962 

0.7734 

0.9420 

0.5947 

0.5427 
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Observed  to  Trained  NNS  Output  for  Exercises  1,2,3  39 

07:43  Wednesday,  January  15,  2003 

The  CORR  Procedure 

Pearson  Correlation  Coefficients,  N  =  139 
Prob  >  | r |  under  HO:  Rho=0 


ITP2 

OTP2 

ITP3 

OTP3 

ITP4 

0TP4 

ITP5 

OTP5 

ITP7 

ITA18 

-0.01960 

-0.06377 

-0.04891 

-0.04596 

-0.01960 

-0.03520 

-0.04891 

-0.04475 

-0.05691 

0.8188 

0.4558 

0.5675 

0.5911 

0.8188 

0.6808 

0.5675 

0.6009 

0.5058 

OTA18 

-0.02194 

-0.05739 

-0.04246 

-0.03912 

-0.02318 

-0.03919 

-0.04536 

-0.04129 

-0.06162 

0.7976 

0.5022 

0.6197 

0.6475 

0.7865 

0.6469 

0.5959 

0.6294 

0.4711 

ITA19 

-0.01264 

-0.03572 

-0.03155 

-0.03293 

-0.01264 

-0.02421 

-0.03155 

-0.03720 

-0.03670 

0.8826 

0.6764 

0.7124 

0.7004 

0.8826 

0.7773 

0.7124 

0.6638 

0.6680 

OTA19 

-0.01536 

-0.04824 

-0.04665 

-0.04836 

-0.01911 

-0.03077 

-0.04223 

-0.05006 

-0.03146 

0.8575 

0.5728 

0.5856 

0.5718 

0.8233 

0.7191 

0.6216 

0.5584 

0.7131 

ITA21 

-0.03691 

-0.23864 

-0.09210 

-0.10682 

-0.03691 

-0.03234 

-0.09210 

-0.09994 

-0.10716 

0.6662 

0.0047 

0.2809 

0.2107 

0.6662 

0.7055 

0.2809 

0.2418 

0.2093 

OTA21 

-0.03925 

-0.24296 

-0.09673 

-0.11145 

-0.03949 

-0.03571 

-0.09743 

-0.10599 

-0.09748 

0.6464 

0.0040 

0.2573 

0.1915 

0.6444 

0.6764 

0.2538 

0.2143 

0.2536 

ITA23 

0.17338 

0.21602 

0.16415 

0.18041 

0.17338 

0.23133 

0.07467 

0.09853 

0.03483 

0.0412 

0.0106 

0.0535 

0.0336 

0.0412 

0.0061 

0.3823 

0.2485 

0.6840 

OTA23 

0.16871 

0.23115 

0.16721 

0.18360 

0.17117 

0.22953 

0.07851 

0.10295 

0.03542 

0.0471 

0.0062 

0.0491 

0.0305 

0.0439 

0.0066 

0.3583 

0.2278 

0.6789 

IM1 

-0.04331 

0.13301 

-0.10805 

-0.11261 

-0.04331 

-0.02963 

-0.10805 

-0.09185 

0.04231 

0.6127 

0.1185 

0.2055 

0.1869 

0.6127 

0.7292 

0.2055 

0.2822 

0.6209 

0M1 

-0.04246 

0.13620 

-0.10083 

-0.10517 

-0.04406 

-0.03090 

-0.09773 

-0.08191 

0.04612 

0.6197 

0.1099 

0.2376 

0.2179 

0.6065 

0.7180 

0.2524 

0.3378 

0.5898 

IM2 

0.03204 

-0.43835 

0.14789 

0.14638 

0.15459 

0.14522 

0.13090 

0.11966 

-0.26279 

0.7081 

<.0001 

0.0823 

0.0855 

0.0692 

0.0881 

0.1246 

0.1606 

0.0018 

0M2 

0.01129 

-0.43662 

0.14001 

0.13864 

0.13656 

0.13066 

0.14534 

0.13436 

-0.26696 

0.8950 

<.0001 

0.1002 

0.1036 

0.1089 

0.1252 

0.0878 

0.1148 

0.0015 

IM3 

-0.03222 

0.30661 

-0.08039 

-0.06992 

-0.03222 

-0.00039 

0.05491 

0.07472 

-0.09353 

0.7065 

0.0002 

0.3469 

0.4134 

0.7065 

0.9964 

0.5209 

0.3820 

0.2735 

0M3 

-0.02047 

0.32194 

-0.06549 

-0.05523 

-0.01969 

0.01392 

0.06009 

0.08105 

-0.08854 

0.8110 

0.0001 

0.4437 

0.5184 

0.8181 

0.8708 

0.4823 

0.3429 

0.3000 

IM4 

-0.01029 

0.16474 

-0.02566 

-0.03115 

-0.01029 

0.01537 

-0.02566 

-0.02420 

-0.02986 

0 . 9043 

0.0526 

0 . 7643 

0.7158 

0 . 9043 

0.8574 

0 . 7643 

0.7774 

0.7271 
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Observed  to  Trained  NNS  Output  for  Exercises  1,2,3  40 

07:43  Wednesday,  January  15,  2003 

The  CORR  Procedure 

Pearson  Correlation  Coefficients,  N  =  139 
Prob  >  | r |  under  HO:  Rho=0 


0TP7 

ITP8 

0TP8 

ITP10 

0TP10 

ITP1 1 

OTP1 1 

ITP12 

OTP12 

ITA18 

-0.04530 

-0.02782 

-0.04116 

-0.08009 

-0.07835 

0.19782 

0.19819 

0.63202 

0.63967 

0.5964 

0.7451 

0.6305 

0.3486 

0.3592 

0.0196 

0.0193 

<.0001 

<.0001 

OTA18 

-0.05025 

-0.02853 

-0.04256 

-0.08787 

-0.08608 

0.19185 

0.19229 

0.64432 

0.65225 

0.5569 

0.7389 

0.6188 

0.3037 

0.3137 

0.0237 

0.0233 

<.0001 

<.0001 

ITA19 

-0.03633 

-0.01795 

-0.02699 

-0.05166 

-0.04922 

-0.02948 

-0.02758 

0.13062 

0.12888 

0.6711 

0.8339 

0.7525 

0.5459 

0.5650 

0.7304 

0.7472 

0.1254 

0.1305 

OTA19 

-0.03120 

-0.01915 

-0.02973 

-0.07604 

-0.07362 

-0.04457 

-0.04284 

0.14164 

0.13933 

0.7154 

0.8230 

0.7283 

0.3736 

0.3890 

0.6024 

0.6165 

0.0963 

0.1019 

ITA21 

-0.09489 

-0.05239 

-0.04369 

-0.15082 

-0.15245 

0.49757 

0.49863 

-0.13329 

-0.12304 

0.2665 

0.5402 

0.6095 

0.0764 

0.0732 

<.0001 

<.0001 

0.1177 

0.1490 

OTA21 

-0.08540 

-0.04263 

-0.03306 

-0.15086 

-0.15254 

0.49300 

0.49441 

-0.13363 

-0.12352 

0.3175 

0.6183 

0.6992 

0.0763 

0.0730 

<.0001 

<.0001 

0.1168 

0.1474 

ITA23 

0.05088 

0.24608 

0.27618 

0.70837 

0.71315 

0.45325 

0.45576 

0.10807 

0.11412 

0.5519 

0.0035 

0.0010 

<.0001 

<.0001 

<.0001 

<.0001 

0.2054 

0.1810 

OTA23 

0.05198 

0.24676 

0.27674 

0.71743 

0.72296 

0.46531 

0.46819 

0.11436 

0.12094 

0.5434 

0.0034 

0.0010 

<.0001 

<.0001 

<.0001 

<.0001 

0.1801 

0.1561 

IM1 

0.04942 

-0.06147 

-0.06920 

-0.17694 

-0.18469 

0.12287 

0.11369 

0.11456 

0.11384 

0.5634 

0.4722 

0.4183 

0.0372 

0.0295 

0.1496 

0.1826 

0.1793 

0.1821 

0M1 

0.05378 

-0.06367 

-0.07130 

-0.18147 

-0.18903 

0.10598 

0.09697 

0.11403 

0.11307 

0.5295 

0.4565 

0.4042 

0.0325 

0.0258 

0.2143 

0.2561 

0.1814 

0.1851 

IM2 

-0.27007 

-0.12848 

-0.11983 

-0.14730 

-0.15906 

-0.08330 

-0.07614 

-0.08101 

-0.08561 

0.0013 

0.1317 

0.1600 

0.0836 

0.0615 

0.3296 

0.3730 

0.3431 

0.3163 

0M2 

-0.27422 

-0.12142 

-0.11394 

-0.15091 

-0.16208 

-0.07137 

-0.06492 

-0.07939 

-0.08389 

0.0011 

0.1545 

0.1817 

0.0762 

0.0566 

0.4037 

0.4477 

0.3529 

0.3262 

I  M3 

-0.07470 

-0.04573 

-0.02145 

0.73371 

0.73920 

0.21223 

0.21655 

0.05643 

0.06014 

0.3821 

0.5930 

0.8021 

<.0001 

<.0001 

0.0121 

0.0105 

0.5094 

0.4819 

0M3 

-0.07010 

-0.02435 

-0.00084 

0.73487 

0.74111 

0.21630 

0.22033 

0.05777 

0.06122 

0.4122 

0.7760 

0.9921 

<.0001 

<.0001 

0.0105 

0.0092 

0.4994 

0.4741 

IM4 

-0.02740 

-0.01460 

-0.02378 

-0.04202 

-0.03750 

-0.10530 

-0.11028 

-0.03714 

-0.04205 

0.7489 

0.8646 

0.7812 

0.6233 

0.6612 

0.2173 

0.1962 

0.6642 

0.6231 
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Observed  to  Trained  NNS  Output  for  Exercises  1,2,3  41 

07:43  Wednesday,  January  15,  2003 

The  CORR  Procedure 

Pearson  Correlation  Coefficients,  N  =  139 
Prob  >  | r |  under  HO:  Rho=0 


ITP15 

0TP15 

ITP22 

OTP22 

ITP23 

0TP23 

ITS1 

OTS1 

ITS2 

ITA18 

-0.07397 

-0.08055 

-0.05303 

-0.05295 

0.02594 

0.02247 

-0.01960 

-0.03299 

-0.04891 

0.3868 

0.3459 

0.5353 

0.5359 

0.7618 

0.7929 

0.8188 

0.6998 

0.5675 

OTA18 

-0.08900 

-0.09536 

-0.03477 

-0.03475 

0.04106 

0.03855 

-0.02112 

-0.03272 

-0.05390 

0.2975 

0.2641 

0.6845 

0.6847 

0.6313 

0.6523 

0.8050 

0.7022 

0.5286 

ITA19 

-0.04771 

-0.04388 

0.41857 

0.41107 

-0.05166 

-0.03970 

-0.01264 

-0.00868 

-0.03155 

0.5771 

0.6080 

<.0001 

<.0001 

0.5459 

0.6426 

0.8826 

0.9192 

0.7124 

OTA19 

-0.07012 

-0.06661 

0.42976 

0.42370 

-0.01821 

-0.00709 

-0.01849 

-0.01466 

-0.02483 

0.4121 

0.4359 

<.0001 

<.0001 

0.8315 

0.9340 

0.8290 

0.8640 

0.7717 

ITA21 

-0.13929 

-0.15175 

-0.09986 

-0.11467 

-0.15082 

-0.16012 

-0.03691 

-0.04356 

-0.09210 

0.1020 

0.0745 

0.2422 

0.1789 

0.0764 

0.0597 

0.6662 

0.6106 

0.2809 

OTA21 

-0.14424 

-0.15671 

-0.10562 

-0.12064 

-0.15324 

-0.16281 

-0.03707 

-0.04465 

-0.08674 

0.0903 

0.0654 

0.2159 

0.1572 

0.0717 

0.0555 

0.6648 

0.6017 

0.3100 

ITA23 

0.40438 

0.41657 

-0.11307 

-0.11930 

-0.17077 

-0.16047 

-0.04180 

-0.04396 

-0.10429 

<.0001 

<.0001 

0.1851 

0.1619 

0 . 0444 

0.0592 

0.6252 

0.6073 

0.2218 

OTA23 

0.42087 

0.43307 

-0.11639 

-0.12293 

-0.16476 

-0.15458 

-0.04191 

-0.04365 

-0.10400 

<.0001 

<.0001 

0.1724 

0.1494 

0.0526 

0.0692 

0.6243 

0.6099 

0.2231 

IM1 

-0.16341 

-0.16848 

0.32028 

0.32660 

0.12436 

0.13814 

0.18822 

0.21460 

0.08450 

0.0546 

0.0474 

0.0001 

<.0001 

0.1447 

0.1049 

0.0265 

0.0112 

0.3226 

0M1 

-0.17044 

-0.17523 

0.33523 

0.3421 1 

0.13802 

0.15197 

0.18492 

0.21288 

0.09017 

0.0449 

0.0391 

<.0001 

<.0001 

0.1052 

0.0741 

0.0293 

0.0119 

0.2912 

IM2 

-0.34157 

-0.34973 

-0.24488 

-0.25571 

-0.36985 

-0.37864 

-0.09052 

-0.10116 

-0.22586 

<.0001 

<.0001 

0.0037 

0.0024 

<.0001 

<.0001 

0.2892 

0.2360 

0.0075 

0M2 

-0.32634 

-0.33504 

-0.25447 

-0.26610 

-0.38208 

-0.39128 

-0.09474 

-0.10501 

-0.23478 

<.0001 

<.0001 

0.0025 

0.0015 

<.0001 

<.0001 

0.2673 

0.2186 

0.0054 

IM3 

0.66723 

0.67870 

0.03858 

0.03259 

-0.13163 

-0.13598 

-0.03222 

-0.04447 

-0.08039 

<.0001 

<.0001 

0.6521 

0.7033 

0.1224 

0.1105 

0.7065 

0.6032 

0.3469 

0M3 

0.67079 

0.68346 

0.04929 

0.04365 

-0.12607 

-0.12950 

-0.01696 

-0.02743 

-0.08720 

<.0001 

<.0001 

0.5645 

0.6099 

0.1392 

0.1287 

0.8429 

0.7486 

0.3074 

IM4 

0.37616 

0.37837 

-0.02782 

-0.01881 

-0.04202 

-0.04484 

-0.01029 

0.00652 

-0.02566 

<.0001 

<.0001 

0.7451 

0.8260 

0.6233 

0.6002 

0 . 9043 

0.9393 

0 . 7643 
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Observed  to  Trained  NNS  Output  for  Exercises  1,2,3  42 

07:43  Wednesday,  January  15,  2003 

The  CORR  Procedure 

Pearson  Correlation  Coefficients,  N  =  139 
Prob  >  | r |  under  HO:  Rho=0 


OTS2 

ITS5 

OTS5 

ITS8 

0TS8 

ITS1 1 

0TS11 

ITS12 

0TS12 

ITA18 

-0.04651 

-0.04891 

-0.05864 

-0.01960 

0.43636 

-0.04448 

-0.02596 

-0.11566 

-0.11117 

0.5867 

0.5675 

0.4929 

0.8188 

<.0001 

0.6031 

0.7616 

0.1751 

0.1926 

OTA18 

-0.05077 

-0.05578 

-0.06529 

-0.02359 

0.45005 

-0.03824 

-0.02017 

-0.11684 

-0.11268 

0.5528 

0.5143 

0.4451 

0.7828 

<.0001 

0.6549 

0.8137 

0.1707 

0.1866 

ITA19 

-0.01921 

-0.03155 

-0.02768 

-0.01264 

-0.08507 

-0.02869 

-0.04021 

-0.07459 

-0.07148 

0.8224 

0.7124 

0 . 7464 

0.8826 

0.3194 

0.7374 

0.6384 

0.3828 

0.4031 

OTA19 

-0.01116 

-0.03081 

-0.02679 

-0.01974 

-0.09422 

-0.04450 

-0.05741 

-0.07868 

-0.07536 

0.8962 

0.7188 

0.7542 

0.8176 

0.2699 

0.6029 

0.5021 

0.3572 

0.3779 

ITA21 

-0.09154 

-0.09210 

-0.08053 

-0.03691 

-0.25958 

-0.08376 

-0.10914 

0.71595 

0.72609 

0.2838 

0.2809 

0.3460 

0.6662 

0.0020 

0.3269 

0.2009 

<.0001 

<.0001 

OTA21 

-0.08653 

-0.09915 

-0.08794 

-0.03925 

-0.26642 

-0.08885 

-0.11484 

0.71767 

0.72782 

0.3111 

0.2455 

0.3033 

0 . 6464 

0.0015 

0.2983 

0.1783 

<.0001 

<■0001 

ITA23 

-0.10719 

0.25363 

0.26080 

0.17338 

-0.00783 

-0.09484 

-0.07786 

-0.01990 

-0.01733 

0.2091 

0.0026 

0.0019 

0.0412 

0.9271 

0.2667 

0.3623 

0.8162 

0.8395 

OTA23 

-0.10679 

0.25498 

0.26187 

0.16781 

0.00156 

-0.08708 

-0.06959 

-0.00994 

-0.00742 

0.2108 

0.0025 

0.0018 

0 . 0483 

0.9854 

0.3081 

0.4156 

0.9075 

0.9309 

IM1 

0.10121 

0.21288 

0.21200 

-0.04331 

0.19804 

-0.09827 

-0.09488 

-0.14710 

-0.15012 

0.2358 

0.0119 

0.0122 

0.6127 

0.0194 

0.2498 

0.2666 

0.0840 

0.0777 

0M1 

0.10755 

0.20448 

0.20347 

-0.04192 

0.19974 

-0.09356 

-0.09042 

-0.16070 

-0.16354 

0.2076 

0.0158 

0.0163 

0.6241 

0.0184 

0.2733 

0.2898 

0.0588 

0.0544 

IM2 

-0.24373 

-0.12393 

-0.11782 

0.15459 

-0.08978 

-0.03855 

-0.05457 

0.16319 

0.17068 

0.0038 

0.1461 

0.1672 

0.0692 

0.2932 

0.6523 

0.5234 

0.0549 

0.0445 

0M2 

-0.25258 

-0.11886 

-0.11104 

0.1481 1 

-0.08760 

-0.02424 

-0.03994 

0.16545 

0.17297 

0.0027 

0.1634 

0.1931 

0.0819 

0.3051 

0.7769 

0.6407 

0.0516 

0.0417 

I  M3 

-0.08270 

0.19020 

0.19385 

-0.03222 

-0.00295 

-0.07311 

-0.05124 

-0.13363 

-0.13463 

0.3331 

0.0249 

0.0222 

0.7065 

0.9725 

0.3924 

0.5492 

0.1168 

0.1141 

0M3 

-0.08893 

0.20511 

0.20878 

-0.03254 

-0.00652 

-0.06536 

-0.04234 

-0.13356 

-0.13426 

0.2979 

0.0154 

0.0136 

0.7037 

0.9392 

0.4446 

0.6207 

0.1170 

0.1151 

IM4 

-0.02590 

-0.02566 

-0.01302 

-0.01029 

-0.07728 

0.62549 

0.61868 

-0.06068 

-0.05752 

0.7622 

0 . 7643 

0.8790 

0 . 9043 

0.3658 

<.0001 

<.0001 

0.4779 

0.5012 
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Observed  to  Trained  NNS  Output  for  Exercises  1,2,3  43 

07:43  Wednesday,  January  15,  2003 

The  CORR  Procedure 

Pearson  Correlation  Coefficients,  N  =  139 
Prob  >  | r |  under  HO:  Rho=0 


ITS15 

OTS15 

ITS18 

0TS18 

ITS19 

0TS19 

ITS20 

0TS20 

ITS22 

ITA18 

-0.03964 

-0.01676 

-0.07707 

-0.06684 

0.04635 

0.03600 

-0.02782 

-0.03226 

-0.02782 

0.6431 

0.8448 

0.3672 

0.4343 

0.5880 

0.6740 

0.7451 

0.7062 

0.7451 

OTA18 

-0.04043 

-0.01764 

-0.08097 

-0.07059 

0.04467 

0.03521 

-0.01920 

-0.02373 

-0.03086 

0.6365 

0.8367 

0.3434 

0.4089 

0.6016 

0.6807 

0.8225 

0.7816 

0.7184 

ITA19 

-0.02557 

-0.02951 

0.11479 

0.11267 

-0.04565 

-0.03983 

-0.01795 

0.00410 

-0.01795 

0.7651 

0.7302 

0.1784 

0.1867 

0.5936 

0.6416 

0.8339 

0.9618 

0.8339 

OTA19 

-0.03202 

-0.03778 

0.10198 

0.09965 

-0.02880 

-0.02224 

-0.00497 

0.01805 

-0.02402 

0.7083 

0.6588 

0.2322 

0.2431 

0.7365 

0.7949 

0.9537 

0.8330 

0.7789 

ITA21 

-0.07464 

-0.08816 

-0.14512 

-0.15281 

0.07725 

0.08991 

-0.05239 

-0.01817 

-0.05239 

0.3825 

0.3020 

0.0883 

0.0725 

0.3661 

0.2925 

0.5402 

0.8319 

0.5402 

OTA21 

-0.08035 

-0.09422 

-0.14798 

-0.15571 

0.07918 

0.09189 

-0.05399 

-0.02012 

-0.05485 

0.3471 

0.2699 

0.0821 

0.0672 

0.3541 

0.2820 

0.5279 

0.8142 

0.5213 

ITA23 

0.24181 

0.27193 

0.56076 

0.57345 

-0.15093 

-0.16048 

-0.05932 

-0.04862 

0.24608 

0.0041 

0.0012 

<.0001 

<.0001 

0.0761 

0.0591 

0.4879 

0.5698 

0.0035 

OTA23 

0.23897 

0.27043 

0.57329 

0.58613 

-0.15422 

-0.16382 

-0.05678 

-0.04584 

0.24152 

0.0046 

0.0013 

<.0001 

<.0001 

0.0699 

0.0540 

0.5067 

0.5921 

0.0042 

IM1 

-0.08757 

-0.08905 

-0.13414 

-0.13623 

-0.08671 

-0.07948 

0.26715 

0.29435 

-0.06147 

0.3053 

0.2972 

0.1154 

0.1098 

0.3101 

0.3524 

0.0015 

0.0004 

0.4722 

0M1 

-0.08653 

-0.08796 

-0.13664 

-0.13878 

-0.07911 

-0.07172 

0.28785 

0.31437 

-0.05420 

0.3111 

0.3032 

0.1087 

0.1032 

0.3546 

0.4015 

0.0006 

0.0002 

0.5262 

IM2 

0.12674 

0.11546 

-0.20674 

-0.21099 

-0.17936 

-0.17727 

-0.12848 

-0.14223 

0.04547 

0.1371 

0.1759 

0.0146 

0.0127 

0.0346 

0.0368 

0.1317 

0.0949 

0.5951 

0M2 

0.12506 

0.11493 

-0.20978 

-0.21426 

-0.18617 

-0.18527 

-0.12233 

-0.13618 

0.05942 

0.1424 

0.1779 

0.0132 

0.0113 

0.0282 

0.0290 

0.1514 

0.1099 

0.4872 

IM3 

0.38956 

0.42516 

0.45376 

0.46271 

-0.11634 

-0.13068 

-0.04573 

-0.05556 

0.14442 

<.0001 

<.0001 

<.0001 

<.0001 

0.1726 

0.1252 

0.5930 

0.5159 

0.0899 

0M3 

0.39136 

0.42722 

0.46258 

0.47196 

-0.12291 

-0.13724 

-0.03734 

-0.04474 

0.15788 

<.0001 

<.0001 

<.0001 

<.0001 

0.1494 

0.1072 

0.6625 

0.6010 

0.0634 

IM4 

-0.02080 

-0.02198 

-0.04044 

-0.04262 

-0.03714 

-0.04209 

-0.01460 

-0.02563 

-0.01460 

0.8080 

0.7973 

0.6365 

0.6184 

0.6642 

0.6228 

0.8646 

0.7645 

0.8646 
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Observed  to  Trained  NNS  Output  for  Exercises  1,2,3  44 

07:43  Wednesday,  January  15,  2003 

The  CORR  Procedure 

Pearson  Correlation  Coefficients,  N  =  139 
Prob  >  | r |  under  HO:  Rho=0 


0TS22 

ITS23 

OTS23 

ITA2 

OTA2 

ITA3 

OTA3 

ITA7 

0TA7 

ITA18 

-0.01761 

0.49908 

0.50672 

-0.01960 

-0.01450 

-0.07079 

-0.06248 

-0.04891 

-0.05341 

0.8370 

<.0001 

<.0001 

0.8188 

0.8655 

0.4076 

0.4650 

0.5675 

0.5323 

OTA18 

-0.02030 

0.52030 

0.52825 

-0.02153 

-0.01537 

-0.06466 

-0.05634 

-0.03171 

-0.03697 

0.8125 

<.0001 

<.0001 

0.8013 

0.8575 

0.4495 

0.5100 

0.7110 

0.6657 

ITA19 

-0.00292 

0.23762 

0.23922 

-0.01264 

0.03631 

-0.04565 

-0.04087 

-0.03155 

-0.01689 

0.9728 

0.0049 

0.0046 

0.8826 

0.6713 

0.5936 

0.6328 

0.7124 

0.8436 

OTA19 

-0.00813 

0.25435 

0.25667 

0.01149 

0.06429 

-0.04289 

-0.03814 

-0.02224 

-0.00727 

0.9244 

0.0025 

0.0023 

0.8932 

0.4521 

0.6161 

0.6558 

0.7950 

0.9323 

ITA21 

-0.07594 

-0.16725 

-0.18357 

-0.03691 

-0.04760 

-0.13329 

-0.13513 

-0.09210 

-0.08821 

0.3743 

0.0491 

0.0305 

0.6662 

0.5779 

0.1177 

0.1127 

0.2809 

0.3018 

OTA21 

-0.07832 

-0.17626 

-0.19281 

-0.03925 

-0.04962 

-0.13509 

-0.13724 

-0.09572 

-0.09182 

0.3594 

0.0379 

0.0230 

0 . 6464 

0.5619 

0.1128 

0.1072 

0.2623 

0.2824 

ITA23 

0.27988 

-0.18937 

-0.19436 

-0.04180 

-0.08354 

-0.15093 

-0.14512 

-0.10429 

-0.10145 

0.0008 

0.0256 

0.0219 

0.6252 

0.3282 

0.0761 

0.0883 

0.2218 

0.2347 

OTA23 

0.27608 

-0.18469 

-0.18975 

-0.04437 

-0.08621 

-0.14621 

-0.14050 

-0.10493 

-0.10298 

0.0010 

0.0295 

0.0253 

0.6040 

0.3129 

0.0859 

0.0990 

0.2189 

0.2277 

IM1 

-0.06453 

0.61320 

0.62502 

0.05959 

0.06940 

0.44742 

0.45989 

0.46962 

0.47616 

0.4504 

<.0001 

<.0001 

0.4859 

0.4169 

<■0001 

<.0001 

<■0001 

<■0001 

0M1 

-0.05654 

0.61940 

0.63081 

0.05976 

0.07086 

0.44883 

0.46130 

0.48826 

0.49508 

0.5085 

<.0001 

<.0001 

0.4847 

0.4072 

<.0001 

<.0001 

<■0001 

<.0001 

IM2 

-0.00031 

-0.41015 

-0.42246 

-0.09052 

-0.13153 

-0.32688 

-0.34356 

-0.22586 

-0.21761 

0.9971 

<.0001 

<.0001 

0.2892 

0.1227 

<.0001 

<.0001 

0.0075 

0.0101 

0M2 

0.01418 

-0.40504 

-0.41752 

-0.09559 

-0.13913 

-0.33181 

-0.34817 

-0.22625 

-0.21899 

0.8684 

<.0001 

<.0001 

0.2630 

0.1024 

<.0001 

<.0001 

0.0074 

0.0096 

I  M3 

0.16219 

-0.06408 

-0.06662 

-0.03222 

-0.05865 

-0.11634 

-0.11014 

-0.08039 

-0.09234 

0.0564 

0.4536 

0.4358 

0.7065 

0.4928 

0.1726 

0.1968 

0.3469 

0.2796 

0M3 

0.17703 

-0.06438 

-0.06628 

-0.03216 

-0.05807 

-0.12420 

-0.11768 

-0.06305 

-0.07481 

0.0371 

0.4514 

0.4382 

0.7071 

0.4971 

0.1452 

0.1677 

0.4609 

0.3814 

IM4 

-0.0141 1 

-0.04660 

-0.03572 

-0.01029 

0.00959 

-0.03714 

-0.02053 

-0.02566 

-0.03203 

0.8691 

0.5859 

0.6764 

0 . 9043 

0.9107 

0.6642 

0.8104 

0 . 7643 

0.7081 
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Observed  to  Trained  NNS  Output  for  Exercises  1,2,3  45 

07:43  Wednesday,  January  15,  2003 

The  CORR  Procedure 

Pearson  Correlation  Coefficients,  N  =  139 
Prob  >  | r |  under  HO:  Rho=0 


ITA10 

0TA10 

ITA1 1 

0TA1 1 

ITA12 

0TA12 

ITA18 

0TA18 

ITA19 

ITA18 

-0.02782 

-0.02491 

-0.05303 

-0.05634 

-0.03420 

-0.01984 

1 .00000 

0.99792 

-0.03420 

0.7451 

0.7710 

0.5353 

0.5101 

0.6894 

0.8167 

<.0001 

0.6894 

OTA18 

-0.03260 

-0.02966 

-0.04271 

-0.04568 

-0.02229 

-0.00800 

0.99792 

1 .00000 

-0.01513 

0.7032 

0.7289 

0.6177 

0.5933 

0.7945 

0.9256 

<.0001 

0.8597 

ITA19 

-0.01795 

-0.02706 

-0.03420 

-0.01894 

-0.02206 

-0.02940 

-0.03420 

-0.01513 

1 .00000 

0.8339 

0.7519 

0.6894 

0.8249 

0.7966 

0.7312 

0.6894 

0.8597 

OTA19 

-0.02757 

-0.03742 

-0.02202 

-0.00761 

-0.03443 

-0.04370 

-0.01357 

0.00748 

0.99483 

0.7473 

0.6618 

0.7970 

0.9292 

0.6874 

0.6094 

0.8740 

0.9304 

<.0001 

ITA21 

-0.05239 

-0.07033 

-0.09986 

-0.12060 

-0.06440 

-0.08290 

-0.09986 

-0.11720 

-0.06440 

0.5402 

0.4107 

0.2422 

0.1573 

0.4513 

0.3319 

0.2422 

0.1694 

0.4513 

OTA21 

-0.05692 

-0.07495 

-0.10834 

-0.12980 

-0.06721 

-0.08644 

-0.10131 

-0.11899 

-0.06848 

0.5057 

0.3806 

0.2043 

0.1278 

0.4318 

0.3116 

0.2354 

0.1630 

0.4231 

ITA23 

-0.05932 

-0.06402 

-0.11307 

-0.09277 

-0.07292 

-0.02778 

-0.11307 

-0.12043 

-0.07292 

0.4879 

0.4540 

0.1851 

0.2774 

0.3936 

0 . 7454 

0.1851 

0.1579 

0.3936 

OTA23 

-0.06043 

-0.06495 

-0.09824 

-0.07801 

-0.06206 

-0.01655 

-0.10879 

-0.11571 

-0.07038 

0.4798 

0.4475 

0.2499 

0.3614 

0.4680 

0.8467 

0.2024 

0.1749 

0.4103 

IM1 

-0.06147 

-0.05273 

0.04191 

0.05897 

-0.07556 

-0.06050 

0.27057 

0.29428 

0.05909 

0.4722 

0.5376 

0.6242 

0.4905 

0.3767 

0.4793 

0.0013 

0.0004 

0.4896 

0M1 

-0.06424 

-0.05537 

0.03624 

0.05383 

-0.06725 

-0.05219 

0.26479 

0.28952 

0.06623 

0.4525 

0.5174 

0.6719 

0.5291 

0.4315 

0.5417 

0.0016 

0.0005 

0.4385 

IM2 

-0.12848 

-0.14305 

-0.24488 

-0.26375 

0.05589 

0.03783 

-0.24488 

-0.26341 

-0.15794 

0.1317 

0.0930 

0.0037 

0.0017 

0.5134 

0.6584 

0.0037 

0.0017 

0.0633 

0M2 

-0.11232 

-0.12593 

-0.23861 

-0.25819 

0.06098 

0.04333 

-0.22952 

-0.24863 

-0.16356 

0.1880 

0.1396 

0.0047 

0.0021 

0.4758 

0.6125 

0.0066 

0.0032 

0 . 0544 

IM3 

-0.04573 

-0.03407 

0.29004 

0.30219 

-0.05621 

-0.02649 

-0.08716 

-0.09458 

0.13300 

0.5930 

0.6905 

0.0005 

0.0003 

0.5110 

0.7569 

0.3076 

0.2681 

0.1186 

0M3 

-0.03790 

-0.02596 

0.30151 

0.31496 

-0.05270 

-0.02210 

-0.08729 

-0.09437 

0.13153 

0.6578 

0.7616 

0.0003 

0.0002 

0.5378 

0.7962 

0.3069 

0.2691 

0.1227 

IM4 

1  .00000 

0.99663 

-0.02782 

-0.00637 

-0.01795 

-0.02464 

-0.02782 

-0.03260 

-0.01795 

<.0001 

<.0001 

0.7451 

0.9407 

0.8339 

0.7734 

0.7451 

0.7032 

0.8339 
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Observed  to  Trained  NNS  Output  for  Exercises  1,2,3  46 

07:43  Wednesday,  January  15,  2003 

The  CORR  Procedure 

Pearson  Correlation  Coefficients,  N  =  139 
Prob  >  | r |  under  HO:  Rho=0 


0TA19 

ITA21 

0TA21 

ITA23 

0TA23 

IM1 

0M1 

IM2 

0M2 

ITA18 

-0.01357 

-0.09986 

-0.10131 

-0.11307 

-0.10879 

0.27057 

0.26479 

-0.24488 

-0.22952 

0.8740 

0.2422 

0.2354 

0.1851 

0.2024 

0.0013 

0.0016 

0.0037 

0.0066 

OTA18 

0.00748 

-0.11720 

-0.11899 

-0.12043 

-0.11571 

0.29428 

0.28952 

-0.26341 

-0.24863 

0.9304 

0.1694 

0.1630 

0.1579 

0.1749 

0.0004 

0.0005 

0.0017 

0.0032 

ITA19 

0.99483 

-0.06440 

-0.06848 

-0.07292 

-0.07038 

0.05909 

0.06623 

-0.15794 

-0.16356 

<.0001 

0.4513 

0.4231 

0.3936 

0.4103 

0.4896 

0.4385 

0.0633 

0 . 0544 

OTA19 

1 .00000 

-0.07567 

-0.07978 

-0.09729 

-0.09478 

0.07723 

0.08592 

-0.18505 

-0.19121 

0.3760 

0.3505 

0.2545 

0.2671 

0.3662 

0.3146 

0.0292 

0.0241 

ITA21 

-0.07567 

1 .00000 

0.99931 

-0.21291 

-0.20752 

-0.22060 

-0.23050 

0.37129 

0.38074 

0.3760 

<.0001 

0.0119 

0.0142 

0.0091 

0.0063 

<.0001 

<.0001 

OTA21 

-0.07978 

0.99931 

1 .00000 

-0.21510 

-0.21000 

-0.23219 

-0.24200 

0.37826 

0.38746 

0.3505 

<.0001 

0.0110 

0.0131 

0.0060 

0.0041 

<.0001 

<.0001 

ITA23 

-0.09729 

-0.21291 

-0.21510 

1 .00000 

0.99915 

-0.06294 

-0.06970 

-0.07704 

-0.08082 

0.2545 

0.0119 

0.0110 

<.0001 

0.4617 

0.4149 

0.3674 

0.3442 

OTA23 

-0.09478 

-0.20752 

-0.21000 

0.99915 

1 .00000 

-0.05978 

-0.06638 

-0.09373 

-0.09655 

0.2671 

0.0142 

0.0131 

<.0001 

0.4845 

0.4375 

0.2724 

0.2582 

IM1 

0.07723 

-0.22060 

-0.23219 

-0.06294 

-0.05978 

1 .00000 

0.99813 

-0.54099 

-0.53359 

0.3662 

0.0091 

0.0060 

0.4617 

0.4845 

<.0001 

<.0001 

<.0001 

0M1 

0.08592 

-0.23050 

-0.24200 

-0.06970 

-0.06638 

0.99813 

1 .00000 

-0.54553 

-0.53881 

0.3146 

0.0063 

0.0041 

0.4149 

0.4375 

<.0001 

<.0001 

<.0001 

IM2 

-0.18505 

0.37129 

0.37826 

-0.07704 

-0.09373 

-0.54099 

-0.54553 

1 .00000 

0.99073 

0.0292 

<.0001 

<.0001 

0.3674 

0.2724 

<.0001 

<.0001 

<.0001 

0M2 

-0.19121 

0.38074 

0.38746 

-0.08082 

-0.09655 

-0.53359 

-0.53881 

0.99073 

1 .00000 

0.0241 

<.0001 

<.0001 

0.3442 

0.2582 

<.0001 

<.0001 

<.0001 

I  M3 

0.11391 

-0.16412 

-0.17002 

0.49283 

0.50568 

-0.19254 

-0.19479 

-0.28604 

-0.27657 

0.1818 

0.0535 

0.0454 

<.0001 

<.0001 

0.0232 

0.0216 

0.0006 

0.0010 

0M3 

0.11265 

-0.17688 

-0.18324 

0.51463 

0.52754 

-0.17748 

-0.18001 

-0.30037 

-0.29162 

0.1867 

0.0373 

0.0308 

<.0001 

<.0001 

0.0366 

0.0340 

0.0003 

0.0005 

IM4 

-0.02757 

-0.05239 

-0.05692 

-0.05932 

-0.06043 

-0.06147 

-0.06424 

-0.12848 

-0.11232 

0.7473 

0.5402 

0.5057 

0.4879 

0.4798 

0.4722 

0.4525 

0.1317 

0.1880 
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Observed  to  Trained  NNS  Output  for  Exercises  1,2,3 

07:43  Wednesday,  January  15, 

The  CORR  Procedure 

Pearson  Correlation  Coefficients,  N  =  139 
Prob  >  | r |  under  HO:  Rho=0 


I  M3 

0M3 

IM4 

0M4 

IM5 

0M5 

ITA18 

-0.08716 

-0.08729 

-0.02782 

-0.06205 

-0.07561 

-0.08818 

0.3076 

0.3069 

0.7451 

0.4681 

0.3763 

0.3020 

OTA18 

-0.09458 

-0.09437 

-0.03260 

-0.06592 

-0.08944 

-0.10245 

0.2681 

0.2691 

0.7032 

0.4407 

0.2951 

0.2301 

ITA19 

0.13300 

0.13153 

-0.01795 

-0.04562 

-0.04877 

-0.04790 

0.1186 

0.1227 

0.8339 

0.5938 

0.5686 

0.5755 

OTA19 

0.11391 

0.11265 

-0.02757 

-0.05787 

-0.05167 

-0.05165 

0.1818 

0.1867 

0.7473 

0.4986 

0.5458 

0.5459 

ITA21 

-0.16412 

-0.17688 

-0.05239 

-0.13258 

-0.14238 

-0.12681 

0.0535 

0.0373 

0.5402 

0.1197 

0.0945 

0.1368 

OTA21 

-0.17002 

-0.18324 

-0.05692 

-0.13914 

-0.13463 

-0.11924 

0.0454 

0.0308 

0.5057 

0.1024 

0.1141 

0.1621 

ITA23 

0.49283 

0.51463 

-0.05932 

-0.04022 

0.36385 

0.38104 

<.0001 

<.0001 

0.4879 

0.6383 

<.0001 

<.0001 

OTA23 

0.50568 

0.52754 

-0.06043 

-0.04124 

0.37475 

0.39208 

<.0001 

<.0001 

0.4798 

0.6298 

<.0001 

<.0001 

IM1 

-0.19254 

-0.17748 

-0.06147 

-0.04654 

-0.16704 

-0.16173 

0.0232 

0.0366 

0.4722 

0.5864 

0.0494 

0.0572 

0M1 

-0.19479 

-0.18001 

-0.06424 

-0.04906 

-0.16462 

-0.15939 

0.0216 

0.0340 

0.4525 

0.5663 

0.0528 

0.0609 

IM2 

-0.28604 

-0.30037 

-0.12848 

-0.10535 

-0.34915 

-0.34768 

0.0006 

0.0003 

0.1317 

0.2171 

<.0001 

<.0001 

0M2 

-0.27657 

-0.29162 

-0.11232 

-0.09127 

-0.34945 

-0.34892 

0.0010 

0.0005 

0.1880 

0.2852 

<.0001 

<.0001 

IM3 

1  .00000 

0.99664 

-0.04573 

-0.00021 

0.30960 

0.32215 

<.0001 

0.5930 

0.9980 

0.0002 

0.0001 

0M3 

0.99664 

1  .00000 

-0.03790 

0.00645 

0.31790 

0.33056 

<.0001 

0.6578 

0.9399 

0.0001 

<.0001 

IM4 

-0.04573 

-0.03790 

1  .00000 

0.96609 

-0.03967 

-0.02458 

0.5930 

0.6578 

<.0001 

0.6429 

0.7740 
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Observed  to  Trained  NNS  Output  for  Exercises  1,2,3  48 

07:43  Wednesday,  January  15,  2003 

The  CORR  Procedure 

Pearson  Correlation  Coefficients,  N  =  139 
Prob  >  | r |  under  HO:  Rho=0 


ITP2 

OTP2 

ITP3 

OTP3 

ITP4 

0TP4 

ITP5 

OTP5 

ITP7 

0M4 

-0.02747 

0.19645 

-0.02400 

-0.02958 

0.01617 

0 . 04454 

0.05601 

0.06085 

-0.03800 

0.7482 

0.0205 

0.7792 

0.7296 

0.8502 

0.6026 

0.5125 

0.4768 

0.6570 

IM5 

-0.02795 

0.50923 

-0.06974 

-0.06982 

-0.02795  - 

0.01982 

-0.06974 

-0.07042 

0.63814 

0.7440 

<.0001 

0.4146 

0.4141 

0.7440 

0.8169 

0.4146 

0.4101 

<.0001 

0M5 

-0.02869 

0.52162 

-0.07882 

-0.07931 

-0.01256  - 

0.00119 

-0.06781 

-0.06745 

0.64104 

0.7374 

<.0001 

0.3563 

0.3534 

0.8834 

0.9889 

0.4277 

0.4302 

<.0001 

Pearson  Correlation 

Coefficients 

,  N  =  139 

Prob  >  | r |  i 

under  HO:  Rho 

'=0 

0TP7 

ITP8 

0TP8 

ITP10 

0TP10 

ITP1 1 

OTP1 1 

ITP12 

OTP12 

0M4 

-0.03786 

-0.04180 

-0.05718 

-0.04092 

-0.03707  - 

0.16277 

-0.16731 

-0.04488 

-0.05127 

0.6581 

0.6251 

0.5038 

0.6325 

0.6649 

0.0556 

0.0490 

0.5999 

0.5489 

IM5 

0.64759 

0.35421 

0.35747 

0.40412 

0.41659 

0.13324 

0.13747 

-0.10093 

-0.09762 

<.0001 

<.0001 

<.0001 

<.0001 

<.0001 

0.1179 

0.1066 

0.2371 

0.2529 

0M5 

0.65306 

0.33721 

0.34426 

0.42151 

0.43320 

0.14285 

0.14753 

-0.10461 

-0.10158 

<.0001 

<.0001 

<.0001 

<.0001 

<.0001 

0.0934 

0.0831 

0.2204 

0.2341 

Pearson  Correlation 

Coefficients 

,  N  =  139 

Prob  >  | r |  i 

under  HO:  Rho 

i=0 

ITP15 

OTP15 

ITP22 

0TP22 

ITP23 

0TP23 

ITS1 

OTS1 

ITS2 

0M4 

0.37133 

0.37528 

-0.06281 

-0.05400 

-0.06793  - 

0.06992 

-0.02747 

-0.01459 

-0.01905 

<.0001 

<.0001 

0.4626 

0.5278 

0.4269 

0.4134 

0.7482 

0.8647 

0.8239 

IM5 

0.44686 

0.45410 

-0.07561 

-0.07461 

-0.11420  - 

0.11936 

-0.02795 

-0.04499 

0.52375 

<.0001 

<.0001 

0.3763 

0.3827 

0.1807 

0.1616 

0 . 7440 

0.5989 

<.0001 

0M5 

0.45886 

0.46644 

-0.07107 

-0.07019 

-0.11882  - 

0.12378 

-0.01463 

-0.03056 

0.53703 

<.0001 

<.0001 

0.4058 

0.4116 

0.1636 

0.1466 

0.8643 

0.7210 

<.0001 
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Observed  to  Trained  NNS  Output  for  Exercises  1,2,3  49 

07:43  Wednesday,  January  15,  2003 

The  CORR  Procedure 

Pearson  Correlation  Coefficients,  N  =  139 
Prob  >  | r |  under  HO:  Rho=0 


0TS2 

ITS5 

0TS5 

ITS8 

0TS8 

ITS1 1 

0TS11 

ITS12 

0TS12 

0M4 

-0.01882 

0.03622 

0.04953 

0.00823 

-0.02725 

0.62542 

0.62405 

-0.11150 

-0.10965 

0.8259 

0.6721 

0.5625 

0.9234 

0.7501 

<■0001 

<.0001 

0.1913 

0.1988 

IM5 

0.52285 

-0.06974 

-0.06895 

-0.02795 

-0.09180  - 

0.06342 

-0.06073 

-0.04796 

-0.05295 

<.0001 

0.4146 

0.4199 

0 . 7440 

0.2824 

0.4582 

0.4776 

0.5751 

0.5359 

0M5 

0.53640 

-0.05903 

-0.05732 

-0.01380 

-0.11183  - 

0.05967 

-0.05784 

-0.03101 

-0.03528 

<.0001 

0.4900 

0.5027 

0.8719 

0.1900 

0.4853 

0.4988 

0.7170 

0.6801 

Pearson  Correlation 

Coefficients 

,  N  =  139 

Prob  >  | r |  i 

jnder  HO:  Rho 

'=0 

ITS15 

0TS15 

ITS18 

0TS18 

ITS19 

0TS19 

ITS20 

0TS20 

ITS22 

0M4 

0.06679 

0.06767 

-0.07680 

-0.07809 

-0.09561  - 

0.10290 

-0.03899 

-0.05475 

0.01028 

0.4347 

0.4287 

0.3688 

0.3609 

0.2629 

0.2280 

0.6486 

0.5221 

0 . 9044 

IM5 

-0.05652 

-0.04620 

0.42447 

0.42462 

-0.10093  - 

0.11262 

-0.03967 

-0.03398 

-0.03967 

0.5087 

0.5891 

<.0001 

<■0001 

0.2371 

0.1868 

0.6429 

0.6913 

0.6429 

0M5 

-0.05342 

-0.04271 

0.42530 

0.42580 

-0.11644  - 

0.12831 

-0.02546 

-0.01561 

-0.02516 

0.5322 

0.6176 

<.0001 

<.0001 

0.1722 

0.1322 

0.7661 

0.8553 

0.7687 

Pearson  Correlation 

Coefficients 

,  N  =  139 

Prob  >  | r |  i 

jnder  HO:  Rho 

'=0 

OTS22 

ITS23 

OTS23 

ITA2 

0TA2 

ITA3 

OTA3 

ITA7 

0TA7 

0M4 

0.00910 

-0.05256 

-0.04017 

-0.02549 

-0.01645 

0.00586 

0.02242 

-0.05864 

-0.06591 

0.9153 

0.5388 

0.6387 

0.7658 

0.8476 

0 . 9454 

0.7934 

0.4929 

0.4407 

IM5 

0.00390 

-0.12664 

-0.12625 

-0.02795 

-0.02038 

0.32853 

0.32882 

-0.06974 

-0.07437 

0.9636 

0.1374 

0.1386 

0 . 7440 

0.8118 

<.0001 

<.0001 

0.4146 

0.3842 

0M5 

0.01816 

-0.13865 

-0.13848 

-0.02994 

-0.02262 

0.33294 

0.33406 

-0.05800 

-0.06258 

0.8320 

0.1036 

0.1040 

0.7265 

0.7915 

<.0001 

<.0001 

0.4976 

0.4643 
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Observed  to  Trained  NNS  Output  for  Exercises  1,2,3  50 

07:43  Wednesday,  January  15,  2003 

The  CORR  Procedure 

Pearson  Correlation  Coefficients,  N  =  139 
Prob  >  | r |  under  HO:  Rho=0 


ITA10 

0TA10 

ITA1 1 

0TA11  ITA12 

0TA12 

ITA18 

OTA18 

ITA19 

0M4 

0.96609 

0.96785 

0.06673 

0.09081  0.00975 

0.00623 

-0.06205 

-0.06592 

-0.04562 

<.0001 

<.0001 

0.4351 

0.2877  0.9093 

0.9420 

0.4681 

0.4407 

0.5938 

IM5 

-0.03967 

-0.03161 

-0.07561  - 

0.06615  -0.04877 

-0.04551 

-0.07561 

-0.08944 

-0.04877 

0.6429 

0.7118 

0.3763 

0.4391  0.5686 

0.5947 

0.3763 

0.2951 

0.5686 

0M5 

-0.02458 

-0.01580 

-0.07634  - 

0.06532  -0.05632 

-0.05207 

-0.08818 

-0.10245 

-0.04790 

0.7740 

0.8536 

0.3717 

0.4448  0.5102 

0.5427 

0.3020 

0.2301 

0.5755 

Pearson  Correlation  Coefficients,  N  =  139 

Prob 

i  >  | r |  under  HO :  Rho=0 

OTA19 

ITA21 

OTA21 

ITA23  0TA23 

IM1 

0M1 

IM2 

0M2 

0M4 

-0.05787 

-0.13258 

-0.13914  - 

0.04022  -0.04124 

-0.04654 

-0.04906 

-0.10535 

-0.09127 

0.4986 

0.1197 

0.1024 

0.6383  0.6298 

0.5864 

0.5663 

0.2171 

0.2852 

IM5 

-0.05167 

-0.14238 

-0.13463 

0.36385  0.37475 

-0.16704 

-0.16462 

-0.34915 

-0.34945 

0.5458 

0.0945 

0.1141 

<.0001  <.0001 

0 . 0494 

0.0528 

<.0001 

<.0001 

0M5 

-0.05165 

-0.12681 

-0.11924 

0.38104  0.39208 

-0.16173 

-0.15939 

-0.34768 

-0.34892 

0.5459 

0.1368 

0.1621 

<.0001  <.0001 

0.0572 

0.0609 

<.0001 

<.0001 

Pearson  Correlation  Coefficients,  N  =  139 

Prob 

i  >  | r |  under  HO :  Rho=0 

IM3 

0M3 

IM4 

0M4 

IM5 

0M5 

0M4  -0.00021 

0.00645 

0.96609  1 

.00000 

-0.04896 

-0.1 

33560 

0 

.9980 

0.9399 

<.0001 

0.5670 

0 

.6774 

IM5  0.30960 

0.31790 

-0.03967  -0 

.04896 

1 .00000 

0.! 

99683 

0 

.0002 

0.0001 

0.6429 

0.5670 

< 

.0001 

0M5  0.32215 

0.33056 

-0.02458  -0 

.03560 

0.99683 

1  .1 

30000 

0 

.0001 

<.0001 

0.7740 

0.6774 

<.0001 
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Appendix  I  -  SAS  Run  Showing  Simple  Statistics  For  All  DV 


SAS  Code: 

options  pageno=l  f ormdlim="-" ; 

TITLE  "Training  Data  123  Regression  Analysis' ; 

DATA  Tngdatal23; 

INPUT  Dayl  Day2  Day3  Day4  Day5  Day6  Day7  Day8  Day9  DaylO  Dayll  Dayl2  Dayl3  Dayl4  Dayl5  Dayl6 
Dayl7  Dayl8  Dayl9  Day20  Hourl  Hour2  Hour3  Hour4  Hour5  Hour6  Hour7  Hour8  Hour9  HourlO  Hourll 
Hourl2  Hourl3  Hourl4  Hourl5  Hourl6  Hourl7  Hourl8  Hourl9  Hour20  Hour21  Hour22  Hour23  Hour24 
TOCAn  TOCDg  TOCBn  TOCBde  ExTypl  ExTyp2  ExTyp3  ExTyp4  ExNol  ExNo2  ExNo3  ExNo4  ExNo5  ExNo6  ExNo7 
ExNo8  ExNo9  ExNolO  ExNoll  ExNol2  ExNol3  ExNol4  ExNol5  ExNol6  ExNol7  ExNol8  ExNol9  ExNo20  SimNo 
SimJANS  SimJAST  SimCBS  SimCCTT  SimSTRM  BOSIntell  BOSIntel2  BOSIntel3  BOSIntel4  BOSManl  BOSMan2 
BOSFS1  BOSFS2  BOSFS3  BOSFS4  BOSFS5  BOSFS6  BOSFS7  BOSFS8  BOSFS9  BOSMAS1  BOSMAS2  BOSMAS3  BOSMAS4 
BOSMAS5  BOSMAS6  BOSMAS7  BOSADA1  BOSADA2  BOSCC1  BOSCC2  BOSCC3  BRIntel  BRMan  BRFS  BRMAS  BRADA 
BRCSS  BRC2  COMAFATDS  COMAMPS  COMASAS  COMCTIS  COMCSSCS  COMFAADC2  COMFBCB2  COMIMETS  COMMCS  COMAMDWS 
COMJSTARS  COMEPLARS  INTELTUAV  INTEL JSTARS  INTELGSR  MISCBattleTiming  MISCBattleTempo 
MISCReconOps  MISCInfoOps  MISCTactics  MISCTOCAct ivity  MOPPLevel  StaffHud  PERSCMDR  PERSXO  PERSBC 
PERSS3  PERSS3NCO  PERSS3RTO  PERSS2  PERSFSO  PERSENGR  PERSALO  PERSS3SGM  SKYCLR  SKYOC  SKYRAIN 
WINDNONE  WINDLOW  WINDMOD  WINDHIGH  LightBG  LightFG  Noise  TEMPDB  TEMPWB  Humidity  TP1  TP2  TP3  TP4 
TP 5  TP 6  TP7  TP 8  TP 9  TP10  TP11  TP12  TP13  TP14  TP15  TP16  TP17  TP18  TP19  TP20  TP21  TP22  TP23  TS1 
TS2  TS3  TS4  TS5  TS6  TS7  TS8  TS9  TS10  TS11  TS12  TS13  TS14  TS15  TS16  TS17  TS18  TS19  TS20  TS21 
TS22  TS23  TA1  TA2  TA3  TA4  TA5  TA6  TA7  TA8  TA9  TA10  TA11  TA12  TA13  TA14  TA15  TA16  TA17  TA18 
TA19  TA20  TA21  TA22  TA23  Missionl  Mission2  Mission3  Mission4  Mission5  Mission6 


CARDS; 
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0  0 

0  0 
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0 
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0 

0 
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0 

0 

0.0 

0.4 

0.0 

0.0 

0.0 

0.0 

*  10  Correlation  Analysis; 
options  pageno=l  f ormdlim="-" ; 
proc  corr  data=Tngdatal23 ; 

TITLE  "Misc  Correlation  for  Exercises  1,2,3"; 

*  Run  corr  on  only  the  output  variables  to  get  simple  statistics  table  showing  the  sum  of  each 
variable; 

*  which  will  indicate  the  number  of  observations  for  that  variable; 


var  TP 1  TP2  TP3  TP4 

TP 5  TP 6  TP7  TP 8  TP 9  TP10  TP11  TP12  TP13  TP14  TP15  TP16  TP17  TP18  TP19  TP20  TP21  TP22  TP23  TS1 
TS2  TS3  TS4  TS5  TS6  TS7  TS8  TS9  TS10  TS11  TS12  TS13  TS14  TS15  TS16  TS17  TS18  TS19  TS20  TS21 
TS22  TS23  TA1  TA2  TA3  TA4  TA5  TA6  TA7  TA8  TA9  TA10  TA11  TA12  TA13  TA14  TA15  TA16  TA17  TA18 
TA19  TA20  TA21  TA22  TA23  Missionl  Mission2  Mission3  Mission4  Mission5  Mission6; 

run; 
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SAS  Output: 

(The  sum  of  the  data  is  shown  in  bold;  those  DV  selected  are  underlined.) 


Misc  Correlation  for  Exercises  1,2,3 
The  CORR  Procedure 


75  Variables: 

TP1 

TP2 

TP3 

TP4 

TP5  TP6 

TP7 

TP8  TP9 

TP1  0 

TP1 1 

TP1  2 

TP1 3 

TPM  TP1 5 

TP1  6 

TPM  TP1 8 

TP1  9 

TP20 

TP21 

TP22 

TP23  TS1 

TS2 

TS3  TS4 

TS5 

TS6 

TS7 

TS8 

TS9  TS1 0 

TS1 1 

TS1 2  TS1 3 

TS1 4 

TS1  5 

TS1  6 

TS1  7 

TS1  8  TS1 9 

TS20 

TS21  TS22 

TS23 

TA1 

TA2 

TA3 

TA4  TA5 

TA6 

TA7  TA8 

TA9 

TA1  0 

TA1 1 

TA1  2 

TA1 3  TAM 

TA1  5 

TA1 6  TAM 

TA1  8 

TA1  9 

TA20 

TA21 

TA22  TA23 

Missionl 

Mission2 

Mission3 

Mission4 

Mission5  Mission6 

Simple  Statistics 

Variable 

N 

Mean 

Std  Dev 

Sum 

Minimum 

Maximum 

TP1 

139 

0 

0 

0 

0 

0 

TP2 

139 

0.00719 

0.08482 

1 .00000 

0 

1 .00000 

TP3 

139 

0.04317 

0.20396 

6.00000 

0 

1 .00000 

TP4 

139 

0.00719 

0.08482 

1 .00000 

0 

1 .00000 

TP5 

139 

0.04317 

0.20396 

6.00000 

0 

1 .00000 

TP6 

139 

0 

0 

0 

0 

0 

TP7 

139 

0.05755 

0.23374 

8.00000 

0 

1 .00000 

TP8 

139 

0.01439 

0.11952 

2.00000 

0 

1 .00000 

TP9 

139 

0 

0 

0 

0 

0 

TP1  0 

139 

0.10791 

0.31139 

15.00000 

0 

1 .00000 

TP1 1 

139 

0.43165 

0.49710 

60.00000 

0 

1 .00000 

TP1  2 

139 

0.08633 

0.28187 

12.00000 

0 

1 .00000 

TP1  3 

139 

0 

0 

0 

0 

0 

TPM 

139 

0 

0 

0 

0 

0 

TP1  5 

139 

0.09353 

0.29222 

13.00000 

0 

1 .00000 

TP16 

139 

0 

0 

0 

0 

0 

TP1  7 

139 

0 

0 

0 

0 

0 

TP1  8 

139 

0 

0 

0 

0 

0 

TP1  9 

139 

0 

0 

0 

0 

0 

TP20 

139 

0.04317 

0.20396 

6.00000 

0 

1 .00000 

TP21 

139 

0.24460 

0.43141 

34.00000 

0 

1 .00000 

TP22 

139 

0.05036 

0.21948 

7.00000 

0 

1 .00000 

TP23 

139 

0.10791 

0.31139 

15.00000 

0 

1 .00000 

TS1 

139 

0.00719 

0.08482 

1 .00000 

0 

1 .00000 

TS2 

139 

0.04317 

0.20396 

6.00000 

0 

1 .00000 

TS3 

139 

0 

0 

0 

0 

0 

TS4 

139 

0 

0 

0 

0 

0 

TS5 

139 

0.04317 

0.20396 

6.00000 

0 

1 .00000 

TS6 

139 

0 

0 

0 

0 

0 

TS7 

139 

0 

0 

0 

0 

0 

TS8 

139 

0.00719 

0.08482 

1 .00000 

0 

1 .00000 

TS9 

139 

0 

0 

0 

0 

0 

TS1  0 

139 

0 

0 

0 

0 

0 

TS1 1 

139 

0.03597 

0.18689 

5.00000 

0 

1 .00000 

TS1  2 

139 

0.20144 

0.40253 

28.00000 

0 

1 .00000 

TS1  3 

139 

0 

0 

0 

0 

0 

TS1  4 

139 

0 

0 

0 

0 

0 

TS1  5 

139 

0.02878 

0.16778 

4.00000 

0 

1 .00000 

TS1  6 

139 

0 

0 

0 

0 

0 
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Simple  Statistics 


Variable 

N 

Mean 

Std  Dev 

Sum 

Minimum 

Maximum 

TS1 7 

139 

0 

0 

0 

0 

0 

TS1 8 

139 

0.10072 

0.30205 

14.00000 

0 

1 .00000 

TS1 9 

139 

0.08633 

0.28187 

12.00000 

0 

1 .00000 

TS20 

139 

0.01439 

0.11952 

2.00000 

0 

1 .00000 

TS21 

139 

0 

0 

0 

0 

0 

TS22 

139 

0.01439 

0.11952 

2.00000 

0 

1 .00000 

TS23 

139 

0.12950 

0.33696 

18.00000 

0 

1 .00000 

TA1 

139 

0 

0 

0 

0 

0 

TA2 

139 

0.00719 

0.08482 

1.00000 

0 

1 .00000 

TA3 

139 

0.08633 

0.28187 

12.00000 

0 

1 .00000 

TA4 

139 

0 

0 

0 

0 

0 

TA5 

139 

0 

0 

0 

0 

0 

TA6 

139 

0 

0 

0 

0 

0 

TA7 

139 

0.04317 

0.20396 

6.00000 

0 

1 .00000 

TA8 

139 

0 

0 

0 

0 

0 

TA9 

139 

0 

0 

0 

0 

0 

TA1  0 

139 

0.01439 

0.11952 

2.00000 

0 

1 .00000 

TA1 1 

139 

0.05036 

0.21948 

7.00000 

0 

1 .00000 

TA1  2 

139 

0.02158 

0.14584 

3.00000 

0 

1 .00000 

TA1  3 

139 

0 

0 

0 

0 

0 

TAM 

139 

0 

0 

0 

0 

0 

TA1  5 

139 

0 

0 

0 

0 

0 

TA1  6 

139 

0 

0 

0 

0 

0 

TA1  7 

139 

0 

0 

0 

0 

0 

TA1  8 

139 

0.05036 

0.21948 

7.00000 

0 

1 .00000 

TA1  9 

139 

0.02158 

0.14584 

3.00000 

0 

1 .00000 

TA20 

139 

0 

0 

0 

0 

0 

TA21 

139 

0.15827 

0.36632 

22.00000 

0 

1 .00000 

TA22 

139 

0 

0 

0 

0 

0 

TA23 

139 

0.19424 

0.39705 

27.00000 

0 

1 .00000 

Missionl 

139 

0.16835 

0.3321 1 

23.40000 

0 

1 .00000 

Mission2 

139 

0.22158 

0.20913 

30.80000 

0 

0.60000 

Mission3 

139 

0.08417 

0.22320 

11.70000 

0 

0.85000 

Mission4 

139 

0.00576 

0.04781 

0.80000 

0 

0.40000 

Mission5 

139 

0.07050 

0.21551 

9.80000 

0 

0.90000 

Mission6 

139 

0 

0 

0 

0 

0 
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Appendix  J  -  First  Linear  Regression  Run  To  Select  IV  For  Inclusion  In  DV  Models 


SAS  Code: 


options  pageno=l  f ormdlim="-" ; 

TITLE  "Training  Data  123  Regression  Analysis'; 

DATA  Tngdatal23; 

INPUT  Dayl  Day2  Day3  Day4  Day5  Day6  Day7  Day8  Day9  DaylO  Dayll  Dayl2  Dayl3  Dayl4  Dayl5  Dayl6 
Dayl7  Dayl8  Dayl9  Day20  Hourl  Hour2  Hour3  Hour4  Hour5  Hour6  Hour7  Hour8  Hour9  HourlO  Hourll 
Hourl2  Hourl3  Hourl4  Hourl5  Hourl6  Hourl7  Hourl8  Hourl9  Hour20  Hour21  Hour22  Hour23  Hour24 
TOCAn  TOCDg  TOCBn  TOCBde  ExTypl  ExTyp2  ExTyp3  ExTyp4  ExNol  ExNo2  ExNo3  ExNo4  ExNo5  ExNo6  ExNo7 
ExNo8  ExNo9  ExNolO  ExNoll  ExNol2  ExNol3  ExNol4  ExNol5  ExNol6  ExNol7  ExNol8  ExNol9  ExNo20  SiraNo 
SimJANS  SimJAST  SimCBS  SimCCTT  SimSTRM  BOSIntell  BOSIntel2  BOSIntel3  BOSIntel4  BOSManl  BOSMan2 
BOSFS1  BOSFS2  BOSFS3  BOSFS4  BOSFS5  BOSFS6  BOSFS7  BOSFS8  BOSFS9  BOSMAS1  BOSMAS2  BOSMAS3  BOSMAS4 
BOSMAS5  BOSMAS6  BOSMAS7  BOSADA1  BOSADA2  BOSCC1  BOSCC2  BOSCC3  BRIntel  BRMan  BRFS  BRMAS  BRADA 
BRCSS  BRC2  COMAFATDS  COMAMPS  COMASAS  COMCTIS  COMCSSCS  COMFAADC2  COMFBCB2  COMIMETS  COMMCS  COMAMDWS 
COMJSTARS  COMEPLARS  INTELTUAV  INTELJSTARS  INTELGSR  MISCBattleTiming  MISCBattleTempo 
MISCReconOps  MISCInfoOps  MISCTactics  MISCTOCActivity  MOPPLevel  StaffHud  PERSCMDR  PERSXO  PERSBC 
PERSS3  PERSS3NCO  PERSS3RTO  PERSS2  PERSFSO  PERSENGR  PERSALO  PERSS3SGM  SKYCLR  SKYOC  SKYRAIN 
WINDNONE  WINDLOW  WINDMOD  WINDHIGH  LightBG  LightFG  Noise  TEMPDB  TEMPWB  Humidity  TP1  TP2  TP3  TP4 
TP 5  TP 6  TP 7  TP 8  TP 9  TP10  TP11  TP12  TP13  TP14  TP15  TP16  TP17  TP18  TP19  TP20  TP21  TP22  TP23  TS1 
TS2  TS3  TS4  TS5  TS6  TS7  TS8  TS9  TS10  TS11  TS12  TS13  TS14  TS15  TS16  TS17  TS18  TS19  TS20  TS21 
TS22  TS23  TA1  TA2  TA3  TA4  TA5  TA6  TA7  TA8  TA9  TA10  TA11  TA12  TA13  TA14  TA15  TA16  TA17  TA18 
TA19  TA20  TA21  TA22  TA23  Missionl  Mission2  Mission3  Mission4  Mission5  Mission6 


CARDS; 
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*  Full  Tng  Regression  Analysis; 

*  Make  regression  run  with  input /independent  variables  remaining  after  pruning  with  JNNS; 

*  and  output/dependent  variables  culled  with  less  than  10  observations  as  shown  in  simple  ; 

*  statistics  table  from  SAS  PROC  CORR  run  above; 
options  pageno=l  f ormdlim="-" ; 

proc  reg  data=Tngdatal23; 

TITLE  "Regression  Analysis  for  Exercises  1,2,3"; 

*  model  TP2=BOSFSl  BOSFS2  BOSFS3  BOSFS7  BOSFS9; 

model  TP10=BOSIntell  BOSIntel2  BOSIntel3  BOSIntel4  BOSManl  BOSMan2  BOSFS2  BOSFS7  BOSFS9  BOSMAS1 
BOSMAS2  BOSMAS4  BOSMAS5  BOSMAS6  BOSMAS7  BOSADA1  BOSADA2  BOSCC1  BOSCC2  BOSCC3  BRIntel  BRMAS 
COMAFATDS 

COMASAS  COMMCS  COMAMDWS  INTELTUAV  MISCBattleTiming  MISCBattleTempo  MISCReconOps  MISCInfoOps 
MISCTOCActivity  MOPPLevel  StaffHud  PERSCMDR  PERSXO  PERSBC  PERSS3  PERSS3NCO  PERSS3RTO  PERSS2 
PERSFSO 

PERSENGR  PERSALO  PERSS3SGM  Humidity; 

model  TPll=BOSIntell  BOSIntel2  BOSIntel3  BOSIntel4  BOSManl  BOSMan2  BOSFS2  BOSFS7  BOSFS9  BOSMAS1 
BOSMAS2  BOSMAS4  BOSMAS5  BOSMAS6  BOSMAS7  BOSADA1  BOSADA2  BOSCC1  BOSCC2  BOSCC3  BRIntel  BRMAS 
COMAFATDS 

COMASAS  COMMCS  COMAMDWS  INTELTUAV  MISCBattleTiming  MISCBattleTempo  MISCReconOps  MISCInfoOps 
MISCTOCActivity  MOPPLevel  StaffHud  PERSCMDR  PERSXO  PERSBC  PERSS3  PERSS3NCO  PERSS3RTO  PERSS2 
PERSFSO 

PERSENGR  PERSALO  PERSS3SGM  Humidity; 

model  TP12=BOSIntell  BOSIntel2  BOSIntel3  BOSIntel4  BOSManl  BOSMan2  BOSFS2  BOSFS7  BOSFS9  BOSMAS1 
BOSMAS2  BOSMAS4  BOSMAS5  BOSMAS6  BOSMAS7  BOSADA1  BOSADA2  BOSCC1  BOSCC2  BOSCC3  BRIntel  BRMAS 
COMAFATDS 
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COMASAS  COMMCS  COMAMDWS  INTELTUAV  MISCBattleTiming  MISCBattleTempo  MISCReconOps  MISCInfoOps 
MISCTOCActivity  MOPPLevel  StaffHud  PERSCMDR  PERSXO  PERSBC  PERSS3  PERSS3NCO  PERSS3RTO  PERSS2 
PERSFSO 

PERSENGR  PERSALO  PERSS3SGM  Humidity; 

model  TP15=BOSIntell  BOSIntel2  BOSIntel3  BOSIntel4  BOSManl  BOSMan2  BOSFS2  BOSFS7  BOSFS9  BOSMAS1 
BOSMAS2  BOSMAS4  BOSMAS5  BOSMAS6  BOSMAS7  BOSADA1  BOSADA2  BOSCC1  BOSCC2  BOSCC3  BRIntel  BRMAS 
COMAFATDS 

COMASAS  COMMCS  COMAMDWS  INTELTUAV  MISCBattleTiming  MISCBattleTempo  MISCReconOps  MISCInfoOps 
MISCTOCActivity  MOPPLevel  StaffHud  PERSCMDR  PERSXO  PERSBC  PERSS3  PERSS3NCO  PERSS3RTO  PERSS2 
PERSFSO 

PERSENGR  PERSALO  PERSS3SGM  Humidity; 

model  TP21=BOSIntell  BOSIntel2  BOSIntel3  BOSIntel4  BOSManl  BOSMan2  BOSFS2  BOSFS7  BOSFS9  BOSMAS1 
BOSMAS2  BOSMAS4  BOSMAS5  BOSMAS6  BOSMAS7  BOSADA1  BOSADA2  BOSCC1  BOSCC2  BOSCC3  BRIntel  BRMAS 
COMAFATDS 

COMASAS  COMMCS  COMAMDWS  INTELTUAV  MISCBattleTiming  MISCBattleTempo  MISCReconOps  MISCInfoOps 
MISCTOCActivity  MOPPLevel  StaffHud  PERSCMDR  PERSXO  PERSBC  PERSS3  PERSS3NCO  PERSS3RTO  PERSS2 
PERSFSO 

PERSENGR  PERSALO  PERSS3SGM  Humidity; 

model  TP23=BOSIntell  BOSIntel2  BOSIntel3  BOSIntel4  BOSManl  BOSMan2  BOSFS2  BOSFS7  BOSFS9  BOSMAS1 
BOSMAS2  BOSMAS4  BOSMAS5  BOSMAS6  BOSMAS7  BOSADA1  BOSADA2  BOSCC1  BOSCC2  BOSCC3  BRIntel  BRMAS 
COMAFATDS 

COMASAS  COMMCS  COMAMDWS  INTELTUAV  MISCBattleTiming  MISCBattleTempo  MISCReconOps  MISCInfoOps 
MISCTOCActivity  MOPPLevel  StaffHud  PERSCMDR  PERSXO  PERSBC  PERSS3  PERSS3NCO  PERSS3RTO  PERSS2 
PERSFSO 

PERSENGR  PERSALO  PERSS3SGM  Humidity; 

model  TS12=BOSIntell  BOSIntel2  BOSIntel3  BOSIntel4  BOSManl  BOSMan2  BOSFS2  BOSFS7  BOSFS9  BOSMAS1 
BOSMAS2  BOSMAS4  BOSMAS5  BOSMAS6  BOSMAS7  BOSADA1  BOSADA2  BOSCC1  BOSCC2  BOSCC3  BRIntel  BRMAS 
COMAFATDS 

COMASAS  COMMCS  COMAMDWS  INTELTUAV  MISCBattleTiming  MISCBattleTempo  MISCReconOps  MISCInfoOps 
MISCTOCActivity  MOPPLevel  StaffHud  PERSCMDR  PERSXO  PERSBC  PERSS3  PERSS3NCO  PERSS3RTO  PERSS2 
PERSFSO 

PERSENGR  PERSALO  PERSS3SGM  Humidity; 

model  TS18=BOSIntell  BOSIntel2  BOSIntel3  BOSIntel4  BOSManl  BOSMan2  BOSFS2  BOSFS7  BOSFS9  BOSMAS1 
BOSMAS2  BOSMAS4  BOSMAS5  BOSMAS6  BOSMAS7  BOSADA1  BOSADA2  BOSCC1  BOSCC2  BOSCC3  BRIntel  BRMAS 
COMAFATDS 

COMASAS  COMMCS  COMAMDWS  INTELTUAV  MISCBattleTiming  MISCBattleTempo  MISCReconOps  MISCInfoOps 
MISCTOCActivity  MOPPLevel  StaffHud  PERSCMDR  PERSXO  PERSBC  PERSS3  PERSS3NCO  PERSS3RTO  PERSS2 
PERSFSO 

PERSENGR  PERSALO  PERSS3SGM  Humidity; 

model  TS1 9=BOSIntell  BOSIntel2  BOSIntel3  BOSIntel4  BOSManl  BOSMan2  BOSFS2  BOSFS7  BOSFS9  BOSMAS1 
BOSMAS2  BOSMAS4  BOSMAS5  BOSMAS6  BOSMAS7  BOSADA1  BOSADA2  BOSCC1  BOSCC2  BOSCC3  BRIntel  BRMAS 
COMAFATDS 

COMASAS  COMMCS  COMAMDWS  INTELTUAV  MISCBattleTiming  MISCBattleTempo  MISCReconOps  MISCInfoOps 
MISCTOCActivity  MOPPLevel  StaffHud  PERSCMDR  PERSXO  PERSBC  PERSS3  PERSS3NCO  PERSS3RTO  PERSS2 
PERSFSO 

PERSENGR  PERSALO  PERSS3SGM  Humidity; 

model  TS23=BOSIntell  BOSIntel2  BOSIntel3  BOSIntel4  BOSManl  BOSMan2  BOSFS2  BOSFS7  BOSFS9  BOSMAS1 
BOSMAS2  BOSMAS4  BOSMAS5  BOSMAS6  BOSMAS7  BOSADA1  BOSADA2  BOSCC1  BOSCC2  BOSCC3  BRIntel  BRMAS 
COMAFATDS 

COMASAS  COMMCS  COMAMDWS  INTELTUAV  MISCBattleTiming  MISCBattleTempo  MISCReconOps  MISCInfoOps 
MISCTOCActivity  MOPPLevel  StaffHud  PERSCMDR  PERSXO  PERSBC  PERSS3  PERSS3NCO  PERSS3RTO  PERSS2 
PERSFSO 

PERSENGR  PERSALO  PERSS3SGM  Humidity; 

model  TA3=BOSIntell  BOSIntel2  BOSIntel3  BOSIntel4  BOSManl  BOSMan2  BOSFS2  BOSFS7  BOSFS9  BOSMAS1 
BOSMAS2  BOSMAS4  BOSMAS5  BOSMAS6  BOSMAS7  BOSADA1  BOSADA2  BOSCC1  BOSCC2  BOSCC3  BRIntel  BRMAS 
COMAFATDS 

COMASAS  COMMCS  COMAMDWS  INTELTUAV  MISCBattleTiming  MISCBattleTempo  MISCReconOps  MISCInfoOps 
MISCTOCActivity  MOPPLevel  StaffHud  PERSCMDR  PERSXO  PERSBC  PERSS3  PERSS3NCO  PERSS3RTO  PERSS2 
PERSFSO 

PERSENGR  PERSALO  PERSS3SGM  Humidity; 

model  TA21=BOSIntell  BOSIntel2  BOSIntel3  BOSIntel4  BOSManl  BOSMan2  BOSFS2  BOSFS7  BOSFS9  BOSMAS1 
BOSMAS2  BOSMAS4  BOSMAS5  BOSMAS6  BOSMAS7  BOSADA1  BOSADA2  BOSCC1  BOSCC2  BOSCC3  BRIntel  BRMAS 
COMAFATDS 

COMASAS  COMMCS  COMAMDWS  INTELTUAV  MISCBattleTiming  MISCBattleTempo  MISCReconOps  MISCInfoOps 
MISCTOCActivity  MOPPLevel  StaffHud  PERSCMDR  PERSXO  PERSBC  PERSS3  PERSS3NCO  PERSS3RTO  PERSS2 
PERSFSO 

PERSENGR  PERSALO  PERSS3SGM  Humidity; 

model  TA23=BOSIntell  BOSIntel2  BOSIntel3  BOSIntel4  BOSManl  BOSMan2  BOSFS2  BOSFS7  BOSFS9  BOSMAS1 
BOSMAS2  BOSMAS4  BOSMAS5  BOSMAS6  BOSMAS7  BOSADA1  BOSADA2  BOSCC1  BOSCC2  BOSCC3  BRIntel  BRMAS 
COMAFATDS 

COMASAS  COMMCS  COMAMDWS  INTELTUAV  MISCBattleTiming  MISCBattleTempo  MISCReconOps  MISCInfoOps 
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MISCTOCActivity  MOPPLevel  StaffHud  PERSCMDR  PERSXO  PERSBC  PERSS3  PERSS3NCO  PERSS3RTO  PERSS2 
PERSFSO 

PERSENGR  PERSALO  PERSS3SGM  Humidity; 

model  Missionl=BOSIntell  BOSIntel2  BOSIntel3  BOSIntel4  BOSManl  BOSMan2  BOSFS2  BOSFS7  BOSFS9 
BOSMAS1 

BOSMAS2  BOSMAS4  BOSMAS5  BOSMAS6  BOSMAS7  BOSADA1  BOSADA2  BOSCC1  BOSCC2  BOSCC3  BRIntel  BRMAS 
COMAFATDS 

COMASAS  COMMCS  COMAMDWS  INTELTUAV  MISCBattleTiming  MISCBattleTempo  MISCReconOps  MISCInfoOps 
MISCTOCActivity  MOPPLevel  StaffHud  PERSCMDR  PERSXO  PERSBC  PERSS3  PERSS3NCO  PERSS3RTO  PERSS2 
PERSFSO 

PERSENGR  PERSALO  PERSS3SGM  Humidity; 

model  Mission2=BOSIntell  BOSIntel2  BOSIntel3  BOSIntel4  BOSManl  BOSMan2  BOSFS2  BOSFS7  BOSFS9 
BOSMAS1 

BOSMAS2  BOSMAS4  BOSMAS5  BOSMAS6  BOSMAS7  BOSADA1  BOSADA2  BOSCC1  BOSCC2  BOSCC3  BRIntel  BRMAS 
COMAFATDS 

COMASAS  COMMCS  COMAMDWS  INTELTUAV  MISCBattleTiming  MISCBattleTempo  MISCReconOps  MISCInfoOps 
MISCTOCActivity  MOPPLevel  StaffHud  PERSCMDR  PERSXO  PERSBC  PERSS3  PERSS3NCO  PERSS3RTO  PERSS2 
PERSFSO 

PERSENGR  PERSALO  PERSS3SGM  Humidity; 

model  Mission3=BOSIntell  BOSIntel2  BOSIntel3  BOSIntel4  BOSManl  BOSMan2  BOSFS2  BOSFS7  BOSFS9 
BOSMAS1 

BOSMAS2  BOSMAS4  BOSMAS5  BOSMAS6  BOSMAS7  BOSADA1  BOSADA2  BOSCC1  BOSCC2  BOSCC3  BRIntel  BRMAS 
COMAFATDS 

COMASAS  COMMCS  COMAMDWS  INTELTUAV  MISCBattleTiming  MISCBattleTempo  MISCReconOps  MISCInfoOps 
MISCTOCActivity  MOPPLevel  StaffHud  PERSCMDR  PERSXO  PERSBC  PERSS3  PERSS3NCO  PERSS3RTO  PERSS2 
PERSFSO 

PERSENGR  PERSALO  PERSS3SGM  Humidity; 

model  Mission5=BOSIntell  BOSIntel2  BOSIntel3  BOSIntel4  BOSManl  BOSMan2  BOSFS2  BOSFS7  BOSFS9 
BOSMAS1 

BOSMAS2  BOSMAS4  BOSMAS5  BOSMAS6  BOSMAS7  BOSADA1  BOSADA2  BOSCC1  BOSCC2  BOSCC3  BRIntel  BRMAS 
COMAFATDS 

COMASAS  COMMCS  COMAMDWS  INTELTUAV  MISCBattleTiming  MISCBattleTempo  MISCReconOps  MISCInfoOps 
MISCTOCActivity  MOPPLevel  StaffHud  PERSCMDR  PERSXO  PERSBC  PERSS3  PERSS3NCO  PERSS3RTO  PERSS2 
PERSFSO 

PERSENGR  PERSALO  PERSS3SGM  Humidity; 

run; 
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SAS  Output: 


Regression  Analysis  for  Exercises  1,2,3 

12:04  Wednesday,  December  18, 

The  REG  Procedure 
Model:  MODEL1 
Dependent  Variable:  TP10 
Analysis  of  Variance 


Sum  of 

Mean 

Source 

DF 

Squares 

Square 

F  Value 

Pr  >  F 

Model 

46 

12 . 02080 

0.26132 

17 . 67 

<■0001 

Error 

92 

1.36049 

0 . 01479 

Corrected  Total 

138 

13.38129 

Root  MSE 

0 . 12161 

R-Square 

0.8983 

Dependent 

Mean 

0 . 10791 

Adj  R-Sq 

0 . 8475 

Coeff  Var 

112 . 68797 

Parameter  Estimates 

Parameter 

Standard 

Variable 

DF 

Estimate 

Error 

t  Value 

Pr  >  1 1  | 

Intercept 

1 

0 . 14508 

0.40682 

0.36 

0 . 7222 

BOSIntell 

1 

0 . 54077 

0 . 15107 

3.58 

0.0006 

BOSIntel2 

1 

0 . 64916 

0.13633 

4 .76 

<.0001 

BOSIntel3 

1 

0.62009 

0.13999 

4 . 43 

<.0001 

BOSIntel4 

1 

0 . 72621 

0 . 17174 

4.23 

<.0001 

BOSManl 

1 

-0 . 14944 

0.13684 

-1.09 

0.2777 

BOSMan2 

1 

0.03083 

0 . 12721 

0.24 

0.8090 

BOSFS2 

1 

0 . 11876 

0.06996 

1 . 70 

0.0930 

BOSFS7 

1 

-0 . 10047 

0.08353 

-1.20 

0.2321 

BOSFS9 

1 

-0.02630 

0 . 07945 

-0.33 

0 . 7414 

BOSMAS1 

1 

0.04637 

0.07388 

0.63 

0.5319 

BOSMAS2 

1 

0.22815 

0.08037 

2 . 84 

0.0056 

BOSMAS4 

1 

0 . 10387 

0 .11042 

0.94 

0.3493 

BOSMAS5 

1 

-0 . 01027 

0 . 04040 

-0.25 

0.8000 

BOSMAS6 

1 

0.62889 

0.20857 

3.02 

0.0033 

BOSMAS7 

1 

0.22498 

0 . 12430 

1.81 

0.0736 

BOSADA1 

1 

-1.24598 

0.39498 

-3.15 

0 . 0022 

BOSADA2 

1 

-1 . 13750 

0 . 37516 

-3.03 

0.0032 

BOSCC1 

1 

-0 . 39701 

0.13599 

-2 . 92 

0 . 0044 

BOSCC2 

1 

-0.50633 

0 . 13754 

-3.68 

0.0004 

BOSCC3 

1 

-0 . 42400 

0 .16674 

-2 . 54 

0 . 0127 

BRIntel 

1 

-0 .01445 

0 .11059 

-0.13 

0.8964 

BRMAS 

1 

-0.33434 

0.23039 

-1 . 45 

0 . 1501 

COMAFATDS 

1 

-0.30881 

0 . 15894 

-1 . 94 

0 . 0551 

COMASAS 

1 

-0.19206 

0 . 18605 

-1.03 

0.3046 

COMMCS 

1 

-0 .06422 

0 .16817 

-0.38 

0 . 7034 

COMAMDWS 

1 

0.09252 

0 .16508 

0.56 

0 . 5765 

INTELTUAV 

1 

-0 .06244 

0.08539 

-0.73 

0 .4665 

MISCBattleTiming 

1 

0 . 40262 

0.36346 

1 . 11 

0.2709 

MISCBattleTempo 

1 

-0 . 33771 

0.27539 

-1.23 

0.2232 

MISCReconOps 

1 

0.04683 

0.06986 

0.67 

0 . 5043 

MISCInfoOps 

1 

-0 . 05997 

0 .09474 

-0.63 

0.5283 

MISCTOCAct ivity 

1 

-0.00732 

0.08631 

-0.08 

0.9326 

MOPPLevel 

1 

2 . 54402 

0 .89515 

2 . 84 

0 . 0055 

Staf fHud 

1 

0 . 01922 

0 . 05865 

0.33 

0 . 7438 

PERSCMDR 

1 

-0 . 08179 

0 . 04877 

-1.68 

0.0969 

PERSXO 

1 

0 . 04952 

0 . 04542 

1.09 

0.2785 

PERSBC 

1 

0.03228 

0 .06556 

0.49 

0.6237 

PERSS3 

1 

0.09948 

0 .06354 

1 . 57 

0.1209 

PERSS3NCO 

1 

-0.02453 

0.06618 

-0.37 

0 . 7117 

PERSS3RTO 

1 

0 . 03501 

0.06963 

0.50 

0.6163 

PERSS2 

1 

0.00925 

0 . 04374 

0.21 

0.8330 

PERSFSO 

1 

-0.03583 

0 . 05340 

-0.67 

0.5039 

PERSENGR 

1 

0.05230 

0.06225 

0.84 

0 .4029 

PERSALO 

1 

-0.06108 

0 . 04341 

-1 . 41 

0 .1627 

PERSS3SGM 

1 

-0.00878 

0 . 04948 

-0.18 

0.8596 

Humidity 

1 

-0 .01476 

0 . 12172 

-0 . 12 

0.9037 

1 

2002 
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Regression  Analysis  for  Exercises  1,2,3 

12:04  Wednesday,  December  18, 

The  REG  Procedure 
Model:  MODEL2 
Dependent  Variable:  TP11 
Analysis  of  Variance 


Sum  of 

Mean 

Source 

DF 

Squares 

Square 

F  Value 

Pr  >  F 

Model 

46 

29.99939 

0.65216 

14 . 63 

<.0001 

Error 

92 

4 . 10133 

0.04458 

Corrected  Total 

138 

34 . 10072 

Root  MSE 

0.21114 

R-Square 

0 . 8797 

Dependent 

Mean 

0 . 43165 

Adj  R-Sq 

0.8196 

Coeff  Var 

48.91388 

Parameter  Estimates 

Parameter 

Standard 

Variable 

DF 

Estimate 

Error 

t  Value 

Pr  >  1 1  | 

Intercept 

1 

-0.50229 

0 . 70635 

-0 . 71 

0 . 4788 

BOSIntell 

1 

0.00360 

0.26230 

0.01 

0.9891 

BOSIntel2 

1 

0.66927 

0.23671 

2.83 

0.0058 

BOSIntel3 

1 

0 . 64752 

0.24306 

2.66 

0.0091 

BOSIntel4 

1 

0 . 53914 

0.29818 

1.81 

0.0739 

BOSManl 

1 

-0 . 04835 

0.23759 

-0.20 

0.8392 

BOSMan2 

1 

0.15882 

0.22087 

0 . 72 

0 . 4739 

BOSFS2 

1 

0.35243 

0 . 12147 

2.90 

0.0046 

BOSFS7 

1 

-0 . 01773 

0 . 14503 

-0 . 12 

0.9030 

BOSFS9 

1 

0.52892 

0 . 13795 

3.83 

0.0002 

BOSMAS1 

1 

-0 . 54929 

0 . 12828 

-4.28 

<.0001 

BOSMAS2 

1 

0.32546 

0 . 13954 

2.33 

0.0219 

BOSMAS4 

1 

0 .09017 

0 .19172 

0 . 47 

0.6393 

BOSMAS5 

1 

-0 . 00344 

0 . 07014 

-0.05 

0.9610 

BOSMAS6 

1 

0 .01149 

0.36213 

0.03 

0 . 9748 

BOSMAS7 

1 

-0.33984 

0.21582 

-1 . 57 

0.1188 

BOSADA1 

1 

-1 . 07905 

0.68580 

-1 . 57 

0 . 1191 

BOSADA2 

1 

-1 . 00264 

0 . 65137 

-1 . 54 

0 . 1272 

BOSCC1 

1 

1.00198 

0.23612 

4.24 

<.0001 

BOSCC2 

1 

0 . 52113 

0.23881 

2 . 18 

0.0316 

BOSCC3 

1 

0 . 51592 

0.28951 

1 . 78 

0.0780 

BRIntel 

1 

-0.38860 

0 .19201 

-2 . 02 

0 . 0459 

BRMAS 

1 

0.25166 

0 . 40001 

0.63 

0.5308 

COMAFATDS 

1 

-0 . 52538 

0.27596 

-1.90 

0.0601 

COMASAS 

1 

0 . 53356 

0.32304 

1 . 65 

0.1020 

COMMCS 

1 

0.29769 

0.29198 

1 . 02 

0.3106 

COMAMDWS 

1 

-0.33594 

0.28663 

-1 .  17 

0.2442 

INTELTUAV 

1 

-0.12996 

0 . 14825 

-0.88 

0.3830 

MISCBattleTiming 

1 

-0 . 74376 

0.63106 

-1.18 

0.2416 

MISCBattleTempo 

1 

0 .46881 

0 . 47815 

0.98 

0 . 3294 

MISCReconOps 

1 

0.22195 

0 . 12130 

1.83 

0 . 0705 

MISCInfoOps 

1 

-0 . 13417 

0 .16449 

-0.82 

0 . 4168 

MISCTOCAct ivity 

1 

-0 . 12704 

0.14986 

-0.85 

0.3988 

MOPPLevel 

1 

0 . 42902 

1 . 55421 

0.28 

0 . 7831 

Staf fHud 

1 

-0.25063 

0.10183 

-2 .46 

0 . 0157 

PERSCMDR 

1 

-0.08363 

0.08468 

-0.99 

0.3259 

PERSXO 

1 

-0 . 17319 

0.07886 

-2.20 

0.0306 

PERSBC 

1 

0.10091 

0 .11384 

0.89 

0 . 3777 

PERSS3 

1 

0.03904 

0 .11032 

0.35 

0 . 7242 

PERSS3NCO 

1 

0.09363 

0 .11490 

0.81 

0 . 4172 

PERSS3RTO 

1 

0.31820 

0.12089 

2.63 

0.0100 

PERSS2 

1 

-0.05202 

0 . 07594 

-0.69 

0 .4950 

PERSFSO 

1 

-0 . 07481 

0 .09271 

-0.81 

0 . 4218 

PERSENGR 

1 

0 . 05818 

0.10808 

0 . 54 

0 .5916 

PERSALO 

1 

-0 . 12234 

0 . 07536 

-1 . 62 

0.1080 

PERSS3SGM 

1 

-0.18962 

0.08591 

-2.21 

0.0298 

Humidity 

1 

-0.13670 

0.21133 

-0.65 

0 . 5194 

3 

2002 


388 


Regression  Analysis  for  Exercises  1,2,3 

12:04  Wednesday,  December  18, 

The  REG  Procedure 
Model:  MODEL3 
Dependent  Variable:  TP12 
Analysis  of  Variance 


Sum  of 

Mean 

Source 

DF 

Squares 

Square 

F  Value 

Pr  >  F 

Model 

46 

9.71978 

0.21130 

15.62 

<.0001 

Error 

92 

1.24425 

0.01352 

Corrected  Total 

138 

10.96403 

Root  MSE 

0.11629 

R-Square 

0.8865 

Dependent 

Mean 

0.08633 

Adj  R-Sq 

0.8298 

Coeff  Var 

134 . 70809 

Parameter  Estimates 

Parameter 

Standard 

Variable 

DF 

Estimate 

Error 

t  Value 

Pr  >  1 1  | 

Intercept 

1 

0 . 84059 

0.38905 

2.16 

0.0333 

BOSIntell 

1 

0 . 04145 

0 . 14447 

0.29 

0 . 7748 

BOSIntel2 

1 

0 . 04924 

0.13038 

0.38 

0.7066 

BOSIntel3 

1 

0 . 05654 

0.13388 

0 . 42 

0.6738 

BOSIntel4 

1 

-0.22509 

0 .16424 

-1 . 37 

0 . 1739 

BOSManl 

1 

-0 .06507 

0.13086 

-0.50 

0 . 6202 

BOSMan2 

1 

0.20691 

0 . 12165 

1 . 70 

0 .0924 

BOSFS2 

1 

-0.25037 

0.06691 

-3.74 

0.0003 

BOSFS7 

1 

0.19653 

0.07988 

2 .46 

0 . 0158 

BOSFS9 

1 

-0.16625 

0 . 07598 

-2 .19 

0.0312 

BOSMAS1 

1 

0 .16507 

0.07066 

2 . 34 

0 . 0217 

BOSMAS2 

1 

0 . 15115 

0.07686 

1 . 97 

0.0522 

BOSMAS4 

1 

0.22585 

0 . 10560 

2 . 14 

0 . 0351 

BOSMAS5 

1 

0 . 00713 

0.03863 

0.18 

0.8539 

BOSMAS6 

1 

-0.92320 

0 .19946 

-4.63 

<.0001 

BOSMAS7 

1 

-0.20452 

0.11888 

-1 . 72 

0.0887 

BOSADA1 

1 

0 . 40077 

0 . 37773 

1.06 

0.2915 

BOSADA2 

1 

0 . 54415 

0 . 35877 

1 . 52 

0.1328 

BOSCC1 

1 

-0.26110 

0 . 13005 

-2 . 01 

0 . 0476 

BOSCC2 

1 

-0 . 08178 

0 . 13154 

-0.62 

0 . 5357 

BOSCC3 

1 

0 . 01942 

0 . 15946 

0 . 12 

0.9033 

BRIntel 

1 

-0.03625 

0 . 10576 

-0.34 

0.7326 

BRMAS 

1 

-0 . 11082 

0.22032 

-0.50 

0.6162 

COMAFATDS 

1 

0 . 37128 

0.15200 

2 . 44 

0 . 0165 

COMASAS 

1 

-0 . 12229 

0 . 17793 

05 

O 

1 

0.4936 

COMMCS 

1 

0.03613 

0.16082 

0.22 

0 . 8228 

COMAMDWS 

1 

-0.23976 

0 . 15787 

-1.52 

0.1323 

INTELTUAV 

1 

0 . 00185 

0.08166 

0.02 

0.9820 

MISCBattleTiming 

1 

-1 . 10196 

0 . 34758 

-3.17 

0.0021 

MISCBattleTempo 

1 

0 . 54116 

0.26336 

2.05 

0 . 0427 

MISCReconOps 

1 

-0 . 10401 

0.06681 

-1 . 56 

0 . 1229 

MISCInfoOps 

1 

-0 .01459 

0.09060 

-0.16 

0 . 8724 

MISCTOCAct ivity 

1 

0 . 05409 

0 . 08254 

0.66 

0.5139 

MOPPLevel 

1 

-1 . 53691 

0.85605 

-1.80 

0 . 0759 

Staf fHud 

1 

0 . 10426 

0.05609 

1.86 

0.0663 

PERSCMDR 

1 

0.02658 

0 . 04664 

0 . 57 

0 . 5702 

PERSXO 

1 

-0 . 15949 

0 . 04344 

-3.67 

0.0004 

PERSBC 

1 

0.13436 

0.06270 

2 . 14 

0.0348 

PERSS3 

1 

0 . 04160 

0 .06077 

0.68 

0 .4954 

PERSS3NCO 

1 

-0.03303 

0.06329 

-0.52 

0.6030 

PERSS3RTO 

1 

0 . 05350 

0.06659 

0.80 

0 . 4238 

PERSS2 

1 

0.03927 

0.04183 

0.94 

0.3503 

PERSFSO 

1 

-0.03763 

0 . 05106 

-0 . 74 

0.4631 

PERSENGR 

1 

0.00216 

0 . 05953 

0.04 

0 . 9711 

PERSALO 

1 

-0.09978 

0 . 04151 

-2 . 40 

0.0182 

PERSS3SGM 

1 

0 . 01815 

0 . 04732 

0.38 

0 . 7022 

Humidity 

1 

-0.27585 

0 .11640 

-2 . 37 

0.0199 
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Regression  Analysis  for  Exercises  1,2,3 

12:04  Wednesday,  December  18, 

The  REG  Procedure 
Model:  MODEL4 
Dependent  Variable:  TP15 
Analysis  of  Variance 


Sum  of 

Mean 

Source 

DF 

Squares 

Square 

F  Value 

Pr  >  F 

Model 

46 

9.65492 

0.20989 

9.07 

<.0001 

Error 

92 

2 . 12925 

0.02314 

Corrected  Total 

138 

11 . 78417 

Root  MSE 

0 . 15213 

R-Square 

0.8193 

Dependent 

Mean 

0.09353 

Adj  R-Sq 

0 . 7290 

Coeff  Var 

162.66386 

Parameter  Estimates 

Parameter 

Standard 

Variable 

DF 

Estimate 

Error 

t  Value 

Pr  >  1 1  | 

Intercept 

1 

-0.33689 

0 . 50894 

-0.66 

0 . 5097 

BOSIntell 

1 

0.25496 

0.18899 

1 . 35 

0.1806 

BOSIntel2 

1 

0 . 37593 

0 . 17056 

2.20 

0.0300 

BOSIntel3 

1 

0 . 32957 

0 . 17513 

1.88 

0.0630 

BOSIntel4 

1 

0 .89149 

0.21485 

4 . 15 

<.0001 

BOSManl 

1 

-0 .46220 

0 . 17119 

-2 . 70 

0.0083 

BOSMan2 

1 

-0 . 13207 

0 . 15914 

-0.83 

0.4088 

BOSFS2 

1 

0.29053 

0 . 08752 

3.32 

0.0013 

BOSFS7 

1 

-0 . 17965 

0 . 10450 

-1 . 72 

0.0890 

BOSFS9 

1 

-0 . 12714 

0.09940 

-1.28 

0.2041 

BOSMAS1 

1 

0.14366 

0.09243 

1 . 55 

0.1236 

BOSMAS2 

1 

0.20747 

0 . 10054 

2.06 

0 . 0419 

BOSMAS4 

1 

0 . 15085 

0 . 13814 

1.09 

0.2777 

BOSMAS5 

1 

-0 . 02449 

0 . 05054 

-0.48 

0.6291 

BOSMAS6 

1 

0.38026 

0.26093 

1 .46 

0 . 1484 

BOSMAS7 

1 

0 . 18426 

0 . 15551 

1.18 

0.2391 

BOSADA1 

1 

-0 . 94101 

0 .49413 

-1.90 

0.0600 

BOSADA2 

1 

-0.78993 

0.46933 

-1.68 

0 .0957 

BOSCC1 

1 

0.21715 

0 . 17013 

1.28 

0.2050 

BOSCC2 

1 

-0.26705 

0 . 17207 

-1 . 55 

0 . 1241 

BOSCC3 

1 

-0.24288 

0.20860 

-1.16 

0.2473 

BRIntel 

1 

0.05993 

0 . 13835 

0.43 

0.6659 

BRMAS 

1 

-0 . 45893 

0.28822 

-1 . 59 

0 . 1148 

COMAFATDS 

1 

-0 . 70435 

0 .19884 

-3.54 

0.0006 

COMASAS 

1 

0.03914 

0.23276 

0 . 17 

0.8668 

COMMCS 

1 

0.21228 

0.21038 

1.01 

0.3156 

COMAMDWS 

1 

0 .19140 

0.20652 

0.93 

0.3565 

INTELTUAV 

1 

-0.07029 

0.10682 

-0.66 

0 . 5122 

MISCBattleTiming 

1 

1 .11481 

0 . 45469 

2 . 45 

0.0161 

MISCBattleTempo 

1 

-0 . 18484 

0 . 34452 

-0 . 54 

0.5929 

MISCReconOps 

1 

0.21084 

0 . 08740 

2 . 41 

0 . 0178 

MISCInfoOps 

1 

-0 . 17734 

0 .11852 

-1 . 50 

0.1380 

MISCTOCAct ivity 

1 

0.01660 

0 . 10798 

0 . 15 

0 . 8781 

MOPPLevel 

1 

1.86639 

1 .11985 

1.67 

0.0990 

Staf fHud 

1 

0.11393 

0 . 07337 

1 . 55 

0.1239 

PERSCMDR 

1 

-0.06885 

0.06101 

-1.13 

0.2620 

PERSXO 

1 

0 . 11758 

0.05682 

2 . 07 

0 . 0413 

PERSBC 

1 

-0.08480 

0.08202 

-1.03 

0.3039 

PERSS3 

1 

-0 . 04372 

0 . 07949 

-0 . 55 

0 . 5837 

PERSS3NCO 

1 

0 . 10142 

0.08279 

1.23 

0.2237 

PERSS3RTO 

1 

0.06973 

0 . 08711 

0.80 

0 . 4255 

PERSS2 

1 

-0 . 04197 

0 . 05472 

-0 . 77 

0 .4450 

PERSFSO 

1 

-0.03333 

0.06680 

-0.50 

0.6190 

PERSENGR 

1 

0 .06453 

0 . 07787 

0.83 

0 .4094 

PERSALO 

1 

0.05808 

0 . 05430 

1 . 07 

0.2876 

PERSS3SGM 

1 

0.03586 

0.06190 

0.58 

0.5638 

Humidity 

1 

0.28759 

0 . 15227 

1.89 

0.0621 
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Regression  Analysis  for  Exercises  1,2,3 

12:04  Wednesday,  December  18, 

The  REG  Procedure 
Model:  MODEL5 
Dependent  Variable:  TP21 
Analysis  of  Variance 


Sum  of 

Mean 

Source 

DF 

Squares 

Square 

F  Value 

Pr  >  F 

Model 

46 

22 . 96379 

0 .49921 

16.89 

<.0001 

Error 

92 

2.71966 

0.02956 

Corrected  Total 

138 

25.68345 

Root  MSE 

0 . 17193 

R-Square 

0 .8941 

Dependent 

Mean 

0.24460 

Adj  R-Sq 

0 . 8412 

Coeff  Var 

70.29098 

Parameter  Estimates 

Parameter 

Standard 

Variable 

DF 

Estimate 

Error 

t  Value 

Pr  >  1 1  | 

Intercept 

1 

0 . 95670 

0 . 57519 

1.66 

0.0997 

BOSIntell 

1 

0 . 37737 

0.21359 

1 . 77 

0.0806 

BOSIntel2 

1 

0 . 05733 

0 .19276 

0.30 

0.7668 

BOSIntel3 

1 

0 . 17225 

0 .19793 

0.87 

0.3864 

BOSIntel4 

1 

0.27331 

0.24281 

1.13 

0.2633 

BOSManl 

1 

0 . 05587 

0.19348 

0.29 

0 . 7734 

BOSMan2 

1 

-0.07696 

0.17986 

-0.43 

0.6697 

BOSFS2 

1 

-0 . 04335 

0.09892 

-0 . 44 

0 . 6622 

BOSFS7 

1 

0.03313 

0 .11810 

0.28 

0 . 7797 

BOSFS9 

1 

-0 . 48379 

0 .11233 

-4.31 

<.0001 

BOSMAS1 

1 

0 . 13448 

0 . 10446 

1.29 

0.2012 

BOSMAS2 

1 

-0.25370 

0.11363 

-2.23 

0.0280 

BOSMAS4 

1 

-0 . 14431 

0 . 15612 

-0.92 

0 . 3577 

BOSMAS5 

1 

-0 . 15955 

0 . 05712 

-2 .79 

0.0063 

BOSMAS6 

1 

0 . 05778 

0.29489 

0.20 

0 . 8451 

BOSMAS7 

1 

0 . 04449 

0 . 17575 

0.25 

0.8007 

BOSADA1 

1 

-0 . 57584 

0 . 55846 

-1.03 

0.3052 

BOSADA2 

1 

-0 . 51010 

0.53042 

-0.96 

0.3387 

BOSCC1 

1 

-0 . 15313 

0 .19228 

-0.80 

0 . 4278 

BOSCC2 

1 

0 .19156 

0 .19447 

0.99 

0 . 3272 

BOSCC3 

1 

0 . 15577 

0.23575 

0.66 

0 . 5104 

BRIntel 

1 

-0.08966 

0.15636 

-0 . 57 

0 .5678 

BRMAS 

1 

-0.35330 

0 . 32574 

-1.08 

0.2809 

COMAFATDS 

1 

0.28279 

0.22472 

1.26 

0.2114 

COMASAS 

1 

-0 . 48925 

0.26305 

-1.86 

0.0661 

COMMCS 

1 

-0.00126 

0.23777 

-0.01 

0.9958 

COMAMDWS 

1 

0 . 37255 

0.23341 

1.60 

0.1139 

INTELTUAV 

1 

0.05969 

0 . 12073 

0.49 

0 . 6222 

MISCBattleTiming 

1 

0.80613 

0 . 51388 

1 . 57 

0 . 1201 

MISCBattleTempo 

1 

-0.88902 

0.38937 

-2.28 

0 . 0247 

MISCReconOps 

1 

-0 . 04034 

0.09878 

-0 . 41 

0.6839 

MISCInfoOps 

1 

0.29941 

0.13395 

2.24 

0 . 0278 

MISCTOCAct ivity 

1 

-0 .19737 

0 . 12203 

-1 . 62 

0.1092 

MOPPLevel 

1 

1 . 17855 

1.26562 

0.93 

0 . 3542 

Staf fHud 

1 

0 . 17889 

0.08292 

2.16 

0.0336 

PERSCMDR 

1 

-0 . 02476 

0.06895 

-0.36 

0 . 7204 

PERSXO 

1 

0 . 01721 

0 .06422 

0.27 

0.7893 

PERSBC 

1 

0.03690 

0.09270 

0.40 

0 . 6915 

PERSS3 

1 

0 . 04410 

0.08984 

0.49 

0 . 6247 

PERSS3NCO 

1 

0 . 07814 

0 .09357 

0.84 

0 .4058 

PERSS3RTO 

1 

-0 . 17552 

0 .09844 

-1 . 78 

0 . 0779 

PERSS2 

1 

0.00886 

0 .06184 

0 . 14 

0.8864 

PERSFSO 

1 

-0.09895 

0 . 07550 

-1.31 

0.1932 

PERSENGR 

1 

-0.08937 

0.08801 

-1 . 02 

0.3126 

PERSALO 

1 

0 . 08558 

0 .06137 

1.39 

0 .1665 

PERSS3SGM 

1 

0 . 04649 

0.06996 

0.66 

0.5080 

Humidity 

1 

-0 .49074 

0 . 17209 

-2.85 

0 . 0054 
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Regression  Analysis  for  Exercises  1,2,3 

12:04  Wednesday,  December  18, 

The  REG  Procedure 
Model:  MODEL6 
Dependent  Variable:  TP23 
Analysis  of  Variance 


Sum  of 

Mean 

Source 

DF 

Squares 

Square 

F  Value 

Pr  >  F 

Model 

46 

10.63384 

0.23117 

7 . 74 

<.0001 

Error 

92 

2 . 74745 

0.02986 

Corrected  Total 

138 

13.38129 

Root  MSE 

0 . 17281 

R-Square 

0 .7947 

Dependent 

Mean 

0 . 10791 

Adj  R-Sq 

0.6920 

Coeff  Var 

160.13808 

Parameter  Estimates 

Parameter 

Standard 

Variable 

DF 

Estimate 

Error 

t  Value 

Pr  >  1 1  | 

Intercept 

1 

1.28361 

0 . 57812 

2.22 

0.0289 

BOSIntell 

1 

-0.21367 

0.21468 

-1.00 

0 . 3222 

BOSIntel2 

1 

-0.00819 

0 .19374 

-0.04 

0.9664 

BOSIntel3 

1 

-0.02938 

0 .19894 

-0 . 15 

0.8829 

BOSIntel4 

1 

-0.18103 

0.24405 

-0 . 74 

0 .4601 

BOSManl 

1 

-0 . 05477 

0 .19446 

-0.28 

0 . 7789 

BOSMan2 

1 

0 . 12427 

0 . 18078 

0.69 

0.4935 

BOSFS2 

1 

-0.08870 

0.09942 

03 

CO 

O 

1 

0.3746 

BOSFS7 

1 

0 . 03971 

0 .11871 

0.33 

0.7388 

BOSFS9 

1 

0 . 10324 

0 .11291 

0.91 

0.3629 

BOSMAS1 

1 

0 . 00257 

0 . 10499 

0.02 

0.9805 

BOSMAS2 

1 

0 . 03594 

0 .11421 

0.31 

0 . 7537 

BOSMAS4 

1 

-0.09720 

0.15692 

-0.62 

0 . 5372 

BOSMAS5 

1 

-0 . 04598 

0 . 05741 

-0.80 

0 . 4253 

BOSMAS6 

1 

0 . 92128 

0.29639 

3.11 

0.0025 

BOSMAS7 

1 

-0 . 17410 

0 . 17665 

03 

03 

O 

1 

0.3269 

BOSADA1 

1 

0 . 30441 

0 . 56130 

0 . 54 

0.5889 

BOSADA2 

1 

0 . 05277 

0 . 53313 

0.10 

0 . 9214 

BOSCC1 

1 

-0 . 11772 

0.19325 

-0.61 

0 . 5439 

BOSCC2 

1 

-0 . 15999 

0 .19546 

-0.82 

0 . 4152 

BOSCC3 

1 

-0.21069 

0.23696 

03 

CO 

O 

1 

0.3762 

BRIntel 

1 

0 . 32372 

0 . 15715 

2.06 

0 . 0422 

BRMAS 

1 

-0 . 92182 

0 . 32740 

-2 . 82 

0.0060 

COMAFATDS 

1 

0 .01105 

0.22586 

0.05 

0.9611 

COMASAS 

1 

-0.38280 

0.26440 

-1 . 45 

0 . 1511 

COMMCS 

1 

-0.21579 

0.23898 

-0.90 

0.3689 

COMAMDWS 

1 

0 . 52055 

0.23460 

2.22 

0.0290 

INTELTUAV 

1 

0 . 05950 

0 . 12134 

0.49 

0.6250 

MISCBattleTiming 

1 

-0 . 50780 

0 . 51650 

-0.98 

0.3281 

MISCBattleTempo 

1 

0.20450 

0.39135 

0.52 

0.6025 

MISCReconOps 

1 

-0.03193 

0.09928 

-0.32 

0 . 7485 

MISCInfoOps 

1 

-0.12388 

0.13463 

-0.92 

0.3599 

MISCTOCAct ivity 

1 

0.00166 

0.12266 

0.01 

0.9892 

MOPPLevel 

1 

-0 . 14931 

1.27207 

-0 . 12 

0.9068 

Staf fHud 

1 

0.18649 

0.08335 

2.24 

0 . 0277 

PERSCMDR 

1 

-0 . 02341 

0.06931 

-0.34 

0.7363 

PERSXO 

1 

-0 . 01727 

0 .06455 

-0.27 

0 . 7897 

PERSBC 

1 

-0.07293 

0 .09317 

-0.78 

0 . 4358 

PERSS3 

1 

0.00848 

0.09030 

0.09 

0 . 9254 

PERSS3NCO 

1 

-0.02105 

0 .09404 

-0.22 

0.8233 

PERSS3RTO 

1 

-0.08529 

0.09895 

-0.86 

0.3909 

PERSS2 

1 

0.03226 

0.06216 

0.52 

0.6050 

PERSFSO 

1 

0.10103 

0 . 07588 

1.33 

0.1863 

PERSENGR 

1 

-0.08887 

0.08846 

-1.00 

0.3177 

PERSALO 

1 

-0.09168 

0.06168 

-1 .49 

0 . 1406 

PERSS3SGM 

1 

0.05030 

0 . 07031 

0 . 72 

0 . 4762 

Humidity 

1 

0.01316 

0 . 17297 

0.08 

0.9395 

11 

2002 
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Regression  Analysis  for  Exercises  1,2,3 

12:04  Wednesday,  December  18, 

The  REG  Procedure 
Model:  MODEL7 
Dependent  Variable:  TS12 
Analysis  of  Variance 


Sum  of 

Mean 

Source 

DF 

Squares 

Square 

F  Value 

Pr  >  F 

Model 

46 

17 . 92488 

0.38967 

8.08 

<.0001 

Error 

92 

4 . 43483 

0.04820 

Corrected  Total 

138 

22 . 35971 

Root  MSE 

0.21956 

R-Square 

0 . 8017 

Dependent 

Mean 

0.20144 

Adj  R-Sq 

0 . 7025 

Coeff  Var 

108.99367 

Parameter  Estimates 

Parameter 

Standard 

Variable 

DF 

Estimate 

Error 

t  Value 

Pr  >  1 1  | 

Intercept 

1 

0 . 78450 

0 . 73451 

1 . 07 

0.2883 

BOSIntell 

1 

-0 .19624 

0.27275 

-0 . 72 

0 . 4737 

BOSIntel2 

1 

0.01680 

0.24615 

0.07 

0 . 9457 

BOSIntel3 

1 

-0 . 15024 

0.25275 

-0.59 

0 . 5537 

BOSIntel4 

1 

-0 . 18814 

0 . 31007 

-0.61 

0 . 5455 

BOSManl 

1 

-0 . 54411 

0.24706 

-2.20 

0.0301 

BOSMan2 

1 

0.06283 

0.22967 

0.27 

0 . 7850 

BOSFS2 

1 

0 . 10227 

0.12631 

0.81 

0 . 4202 

BOSFS7 

1 

0.09598 

0.15082 

0.64 

0.5261 

BOSFS9 

1 

1 .  12024 

0 . 14345 

7 . 81 

<.0001 

BOSMAS1 

1 

-0.48360 

0.13339 

-3.63 

0.0005 

BOSMAS2 

1 

-0 .19611 

0 . 14510 

-1 . 35 

0 . 1798 

BOSMAS4 

1 

-0.30868 

0.19936 

-1 . 55 

0.1250 

BOSMAS5 

1 

0 . 08654 

0 . 07294 

1.19 

0.2385 

BOSMAS6 

1 

0 . 00107 

0 . 37657 

0.00 

0 . 9977 

BOSMAS7 

1 

-0.24532 

0.22443 

-1.09 

0.2772 

BOSADA1 

1 

1 . 07420 

0 . 71313 

1 . 51 

0 . 1354 

BOSADA2 

1 

0.33059 

0 . 67733 

0.49 

0.6267 

BOSCC1 

1 

-0.39914 

0.24553 

-1.63 

0 . 1074 

BOSCC2 

1 

-0.38024 

0.24833 

-1 . 53 

0 . 1292 

BOSCC3 

1 

-0 . 58446 

0 . 30105 

-1 . 94 

0 . 0553 

BRIntel 

1 

0 . 17590 

0.19966 

0.88 

0.3806 

BRMAS 

1 

-0.41638 

0 .41596 

-1.00 

0 . 3194 

COMAFATDS 

1 

0.02695 

0.28696 

0.09 

0 . 9254 

COMASAS 

1 

-0.28418 

0.33591 

-0.85 

0.3998 

COMMCS 

1 

0 . 03581 

0.30362 

0 . 12 

0.9064 

COMAMDWS 

1 

0 . 30105 

0.29806 

1.01 

0 . 3151 

INTELTUAV 

1 

-0 . 12468 

0 . 15416 

-0.81 

0 . 4208 

MISCBattleTiming 

1 

0.24532 

0.65621 

0.37 

0 . 7094 

MISCBattleTempo 

1 

-0.18668 

0 .49721 

-0.38 

0 . 7082 

MISCReconOps 

1 

0 . 05750 

0 . 12614 

0.46 

0 . 6495 

MISCInfoOps 

1 

-0.01582 

0 . 17105 

-0.09 

0.9265 

MISCTOCAct ivity 

1 

-0 . 10997 

0 . 15583 

-0 . 71 

0 . 4822 

MOPPLevel 

1 

-1 . 02779 

1 . 61616 

-0.64 

0 . 5264 

Staf fHud 

1 

-0.05690 

0.10589 

-0 . 54 

0.5923 

PERSCMDR 

1 

-0 . 07428 

0.08805 

-0.84 

0 .4011 

PERSXO 

1 

-0.10382 

0.08200 

-1.27 

0.2087 

PERSBC 

1 

-0 . 01374 

0 .11837 

-0 . 12 

0.9078 

PERSS3 

1 

-0 . 07720 

0 .11472 

-0.67 

0 . 5027 

PERSS3NCO 

1 

0 .16922 

0 .11948 

1 . 42 

0.1601 

PERSS3RTO 

1 

-0.09952 

0 . 12571 

-0.79 

0.4306 

PERSS2 

1 

0 . 14655 

0 . 07897 

1.86 

0.0667 

PERSFSO 

1 

0 .09744 

0 .09641 

1.01 

0 . 3148 

PERSENGR 

1 

-0.19661 

0 .11239 

-1 . 75 

0.0836 

PERSALO 

1 

-0 . 07937 

0 . 07837 

-1.01 

0.3138 

PERSS3SGM 

1 

0 .01491 

0.08933 

0 . 17 

0.8678 

Humidity 

1 

-0 . 14996 

0.21976 

-0.68 

0.4967 

13 
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Regression  Analysis  for  Exercises  1,2,3 

12:04  Wednesday,  December  18, 

The  REG  Procedure 
Model:  MODEL8 
Dependent  Variable:  TS18 
Analysis  of  Variance 


Sum  of 

Mean 

Source 

DF 

Squares 

Square 

F  Value 

Pr  >  F 

Model 

46 

10.36896 

0.22541 

9.34 

<.0001 

Error 

92 

2.22097 

0.02414 

Corrected  Total 

138 

12.58993 

Root  MSE 

0 . 15537 

R-Square 

0.8236 

Dependent 

Mean 

0 . 10072 

Adj  R-Sq 

0 . 7354 

Coeff  Var 

154.26386 

Parameter  Estimates 

Parameter 

Standard 

Variable 

DF 

Estimate 

Error 

t  Value 

Pr  >  1 1  | 

Intercept 

1 

-0 . 84872 

0 . 51979 

-1.63 

0 . 1059 

BOSIntell 

1 

0.33014 

0.19302 

1 . 71 

0.0906 

BOSIntel2 

1 

0.31382 

0 . 17419 

1.80 

0 . 0749 

BOSIntel3 

1 

0.23324 

0.17886 

1.30 

0 .1955 

BOSIntel4 

1 

0 . 34978 

0.21943 

1 . 59 

0 . 1143 

BOSManl 

1 

0.03801 

0 . 17484 

0.22 

0 . 8284 

BOSMan2 

1 

-0.21791 

0.16253 

-1 . 34 

0.1833 

BOSFS2 

1 

0.28116 

0.08939 

3.15 

0 . 0022 

BOSFS7 

1 

-0.20851 

0.10673 

-1 . 95 

0.0538 

BOSFS9 

1 

-0.08953 

0 . 10151 

-0.88 

0.3801 

BOSMAS1 

1 

0 . 03975 

0 .09440 

0 . 42 

0 . 6747 

BOSMAS2 

1 

0.19679 

0.10268 

1 . 92 

0 . 0584 

BOSMAS4 

1 

0.02258 

0 . 14108 

0.16 

0 . 8732 

BOSMAS5 

1 

-0.03603 

0.05162 

-0.70 

0.4869 

BOSMAS6 

1 

-0 . 55105 

0.26649 

-2 . 07 

0 . 0415 

BOSMAS7 

1 

0.05988 

0.15882 

0.38 

0 . 7070 

BOSADA1 

1 

0 . 17804 

0 . 50467 

0.35 

0 . 7251 

BOSADA2 

1 

0.13023 

0 . 47933 

0.27 

0 . 7865 

BOSCC1 

1 

0 . 55628 

0 . 17375 

3.20 

0.0019 

BOSCC2 

1 

-0 . 07004 

0 . 17574 

-0.40 

0.6912 

BOSCC3 

1 

0 . 00058764 

0.21305 

0.00 

0.9978 

BRIntel 

1 

0 . 04465 

0 . 14129 

0.32 

0 . 7527 

BRMAS 

1 

-0.23591 

0.29436 

-0.80 

0 . 4250 

COMAFATDS 

1 

-0.68420 

0.20307 

-3.37 

0.0011 

COMASAS 

1 

0.23344 

0.23772 

0.98 

0 . 3287 

COMMCS 

1 

-0 . 08051 

0.21486 

-0.37 

0.7088 

COMAMDWS 

1 

0 . 32152 

0.21093 

1.52 

0.1309 

INTELTUAV 

1 

-0.09664 

0.10910 

-0.89 

0.3780 

MISCBattleTiming 

1 

1 . 58978 

0 .46438 

3.42 

0.0009 

MISCBattleTempo 

1 

-0 . 34945 

0 . 35186 

-0.99 

0.3233 

MISCReconOps 

1 

0.19662 

0.08926 

2.20 

0.0301 

MISCInfoOps 

1 

-0.27052 

0 . 12105 

-2.23 

0 . 0278 

MISCTOCAct ivity 

1 

0.06932 

0 .11028 

0.63 

0.5312 

MOPPLevel 

1 

-0.69359 

1 . 14372 

-0.61 

0 . 5457 

Staf fHud 

1 

-0.00372 

0 . 07494 

-0.05 

0.9605 

PERSCMDR 

1 

-0.06091 

0.06231 

-0.98 

0.3309 

PERSXO 

1 

0 . 12447 

0.05803 

2 . 14 

0.0346 

PERSBC 

1 

-0.07463 

0 . 08377 

-0.89 

0 . 3753 

PERSS3 

1 

-0 . 0  6553 

0.08119 

-0.81 

0 . 4217 

PERSS3NCO 

1 

0 .11771 

0 . 08455 

1.39 

0 .1672 

PERSS3RTO 

1 

0.00989 

0.08896 

0.11 

0 . 9117 

PERSS2 

1 

-0 . 04742 

0 . 05589 

-0.85 

0.3984 

PERSFSO 

1 

-0 . 00512 

0.06822 

-0.08 

0 . 9404 

PERSENGR 

1 

0.08878 

0 . 07953 

1 . 12 

0.2672 

PERSALO 

1 

0.02813 

0 . 05546 

0 . 51 

0.6132 

PERSS3SGM 

1 

0 . 00485 

0.06322 

0.08 

0.9390 

Humidity 

1 

0.20854 

0 . 15552 

1 . 34 

0.1833 

15 

2002 
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Regression  Analysis  for  Exercises  1,2,3 

12:04  Wednesday,  December  18, 

The  REG  Procedure 
Model:  MODEL9 
Dependent  Variable:  TS19 
Analysis  of  Variance 


Sum  of 

Mean 

Source 

DF 

Squares 

Square 

F  Value 

Pr  >  F 

Model 

46 

8.53288 

0.18550 

7 . 02 

<.0001 

Error 

92 

2 . 43115 

0.02643 

Corrected  Total 

138 

10.96403 

Root  MSE 

0.16256 

R-Square 

0 . 7783 

Dependent 

Mean 

0.08633 

Adj  R-Sq 

0 . 6674 

Coeff  Var 

188.29798 

Parameter  Estimates 

Parameter 

Standard 

Variable 

DF 

Estimate 

Error 

t  Value 

Pr  >  1 1  | 

Intercept 

1 

0 .49914 

0 . 54383 

0.92 

0.3611 

BOSIntell 

1 

0 . 05418 

0.20195 

0.27 

0 . 7891 

BOSIntel2 

1 

-0.06022 

0 . 18225 

-0.33 

0 . 7418 

BOSIntel3 

1 

0.08022 

0 . 18714 

0.43 

0.6692 

BOSIntel4 

1 

-0.08120 

0.22957 

-0.35 

0 . 7244 

BOSManl 

1 

0.09790 

0.18293 

0 . 54 

0.5938 

BOSMan2 

1 

0 . 04351 

0 . 17005 

0.26 

0.7986 

BOSFS2 

1 

-0.06087 

0.09352 

-0.65 

0 . 5167 

BOSFS7 

1 

0 . 01814 

0 .11166 

0.16 

0 . 8713 

BOSFS9 

1 

-0.03919 

0.10621 

-0.37 

0 . 7130 

BOSMAS1 

1 

-0 . 01281 

0.09876 

-0.13 

0.8970 

BOSMAS2 

1 

0 . 00773 

0 . 10743 

0.07 

0 . 9428 

BOSMAS4 

1 

0.21517 

0 . 14761 

1 .46 

0 . 1483 

BOSMAS5 

1 

-0 . 07375 

0 . 05400 

-1 . 37 

0 . 1754 

BOSMAS6 

1 

0.99070 

0.27881 

3.55 

0.0006 

BOSMAS7 

1 

-0 . 01945 

0 .16617 

-0 . 12 

0 . 9071 

BOSADA1 

1 

-0 . 30974 

0.52800 

-0.59 

0 . 5589 

BOSADA2 

1 

-0.34367 

0 . 50150 

03 

UD 

O 

1 

0 .4949 

BOSCC1 

1 

0 . 02322 

0 . 18179 

0.13 

0.8987 

BOSCC2 

1 

0.01993 

0.18386 

0.11 

0.9139 

BOSCC3 

1 

0 . 05427 

0.22290 

0.24 

0.8082 

BRIntel 

1 

-0 . 03954 

0 . 14783 

-0.27 

0 . 7897 

BRMAS 

1 

-0 . 17554 

0 . 30797 

-0 . 57 

0 . 5701 

COMAFATDS 

1 

-0.00998 

0.21246 

-0.05 

0.9626 

COMASAS 

1 

0 . 35579 

0.24871 

1.43 

0 . 1559 

COMMCS 

1 

-0 . 05135 

0.22480 

-0.23 

0.8198 

COMAMDWS 

1 

0 . 04087 

0.22068 

0.19 

0 . 8535 

INTELTUAV 

1 

0.12106 

0 .11414 

1.06 

0.2916 

MISCBattleTiming 

1 

-0 . 71202 

0 . 48586 

-1 . 47 

0 . 1462 

MISCBattleTempo 

1 

0 . 57430 

0.36813 

1 . 56 

0 . 1222 

MISCReconOps 

1 

-0 . 04252 

0.09339 

-0.46 

0.6500 

MISCInfoOps 

1 

-0 . 18145 

0 . 12664 

-1.43 

0 . 1553 

MISCTOCAct ivity 

1 

0.16689 

0 .11538 

1 . 45 

0 . 1515 

MOPPLevel 

1 

-0.39983 

1 .19661 

-0.33 

0.7390 

Staf fHud 

1 

0 . 03448 

0 . 07840 

0 . 44 

0.6611 

PERSCMDR 

1 

-0 . 04431 

0 .06519 

-0.68 

0 .4985 

PERSXO 

1 

-0.08812 

0.06072 

-1 . 45 

0 . 1501 

PERSBC 

1 

0.10952 

0 . 08764 

1.25 

0.2146 

PERSS3 

1 

0.01892 

0 . 08494 

0.22 

0 . 8242 

PERSS3NCO 

1 

0.00927 

0.08846 

0.10 

0.9168 

PERSS3RTO 

1 

0 .09717 

0.09308 

1 . 04 

0.2992 

PERSS2 

1 

-0 . 17163 

0 . 05847 

-2 . 94 

0 . 0042 

PERSFSO 

1 

-0 . 02227 

0 . 07138 

-0.31 

0 . 7558 

PERSENGR 

1 

0.04288 

0.08321 

0.52 

0.6076 

PERSALO 

1 

-0 .01497 

0.05802 

-0.26 

0 .7970 

PERSS3SGM 

1 

-0.03661 

0 .06614 

-0 . 55 

0.5812 

Humidity 

1 

-0.00176 

0 .16271 

-0.01 

0 . 9914 

17 

2002 
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Regression  Analysis  for  Exercises  1,2,3 

12:04  Wednesday,  December  18, 

The  REG  Procedure 
Model:  MODELIO 
Dependent  Variable:  TS23 
Analysis  of  Variance 


Sum  of 

Mean 

Source 

DF 

Squares 

Square 

F  Value 

Pr  >  F 

Model 

46 

12 . 31885 

0.26780 

7.35 

<.0001 

Error 

92 

3.35022 

0.03642 

Corrected  Total 

138 

15.66906 

Root  MSE 

0.19083 

R-Square 

0.7862 

Dependent 

Mean 

0.12950 

Adj  R-Sq 

0.6793 

Coeff  Var 

147 . 36176 

Parameter  Estimates 

Parameter 

Standard 

Variable 

DF 

Estimate 

Error 

t  Value 

Pr  >  1 1  | 

Intercept 

1 

-0 . 84441 

0.63840 

-1 . 32 

0.1892 

BOSIntell 

1 

0.24383 

0.23707 

1.03 

0.3064 

BOSIntel2 

1 

0.61029 

0.21394 

2.85 

0 . 0054 

BOSIntel3 

1 

0.62329 

0.21968 

2 . 84 

0.0056 

BOSIntel4 

1 

0 . 45900 

0.26950 

1 . 70 

0.0919 

BOSManl 

1 

0 .11169 

0.21474 

0.52 

0.6042 

BOSMan2 

1 

-0 .41947 

0.19962 

-2 . 10 

0.0383 

BOSFS2 

1 

0.09036 

0 . 10979 

0.82 

0 . 4126 

BOSFS7 

1 

0 . 07810 

0.13108 

0.60 

0 . 5528 

BOSFS9 

1 

-0.27244 

0 . 12468 

-2 .19 

0 . 0314 

BOSMAS1 

1 

0.01253 

0 .11594 

0.11 

0 . 9142 

BOSMAS2 

1 

-0.03972 

0 . 12611 

i — 1 
00 

o 

1 

0 . 7535 

BOSMAS4 

1 

0 . 40848 

0 . 17328 

2.36 

0.0205 

BOSMAS5 

1 

0 . 00051952 

0.06339 

0.01 

0.9935 

BOSMAS6 

1 

0 . 31012 

0.32730 

0.95 

0 . 3459 

BOSMAS7 

1 

-0.26211 

0 .19506 

-1 . 34 

0.1823 

BOSADA1 

1 

-1.12680 

0.61982 

-1 . 82 

0 . 0723 

BOSADA2 

1 

-0 . 88274 

0 . 58871 

-1 . 50 

0 . 1372 

BOSCC1 

1 

0.20126 

0.21340 

0.94 

0.3481 

BOSCC2 

1 

0 . 47727 

0.21584 

2.21 

0.0295 

BOSCC3 

1 

0 . 44537 

0.26166 

1 . 70 

0.0921 

BRIntel 

1 

-0.09304 

0 . 17354 

-0 . 54 

0.5932 

BRMAS 

1 

-0.04933 

0 . 36153 

-0 . 14 

0.8918 

COMAFATDS 

1 

0 . 17445 

0.24941 

0.70 

0.4860 

COMASAS 

1 

-0 .06745 

0.29196 

-0.23 

0 . 8178 

COMMCS 

1 

-0 . 13614 

0.26389 

-0.52 

0.6072 

COMAMDWS 

1 

0.01329 

0.25906 

0.05 

0.9592 

INTELTUAV 

1 

-0 . 01645 

0.13399 

-0 . 12 

0.9026 

MISCBattleTiming 

1 

-1.36692 

0 . 57035 

-2 . 40 

0.0186 

MISCBattleTempo 

1 

0.21908 

0 . 43215 

0 . 51 

0.6134 

MISCReconOps 

1 

0 . 02745 

0.10963 

0.25 

0.8029 

MISCInfoOps 

1 

0 .41538 

0 . 14867 

2 .79 

0.0063 

MISCTOCAct ivity 

1 

-0.21043 

0 . 13544 

-1 . 55 

0 . 1237 

MOPPLevel 

1 

2 . 45929 

1 . 40470 

1 . 75 

0.0833 

Staf fHud 

1 

-0 . 17017 

0.09204 

-1 . 85 

0 .0677 

PERSCMDR 

1 

-0.07681 

0 . 07653 

o 

o 

I — 1 

1 

0.3182 

PERSXO 

1 

-0.09053 

0 . 07127 

-1.27 

0.2073 

PERSBC 

1 

0 .09211 

0.10288 

0.90 

0.3730 

PERSS3 

1 

0 . 04678 

0 .09971 

0 . 47 

0 . 6401 

PERSS3NCO 

1 

-0.00436 

0 . 10385 

"st1 

O 

o 

1 

0.9666 

PERSS3RTO 

1 

0.33330 

0.10926 

3.05 

0.0030 

PERSS2 

1 

0.02409 

0.06864 

0.35 

0.7265 

PERSFSO 

1 

-0 . 07941 

0.08379 

LD 

O'! 

O 

1 

0 . 3458 

PERSENGR 

1 

0.08878 

0.09768 

0.91 

0.3658 

PERSALO 

1 

0.08490 

0.06811 

1.25 

0.2158 

PERSS3SGM 

1 

0 . 03015 

0 . 07764 

0.39 

0.6986 

Humidity 

1 

0 . 48471 

0 .19101 

2 . 54 

0.0128 
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Regression  Analysis  for  Exercises  1,2,3 

12:04  Wednesday,  December  18, 

The  REG  Procedure 
Model:  MODEL11 
Dependent  Variable:  TA3 
Analysis  of  Variance 


Sum  of 

Mean 

Source 

DF 

Squares 

Square 

F  Value 

Pr  >  F 

Model 

46 

10 . 03424 

0.21814 

21.58 

<.0001 

Error 

92 

0.92979 

0.01011 

Corrected  Total 

138 

10.96403 

Root  MSE 

0 . 10053 

R-Square 

0.9152 

Dependent 

Mean 

0.08633 

Adj  R-Sq 

0 . 8728 

Coeff  Var 

116.44820 

Parameter  Estimates 

Parameter 

Standard 

Variable 

DF 

Estimate 

Error 

t  Value 

Pr  >  1 1  | 

Intercept 

1 

-0.23239 

0.33632 

-0.69 

0 .4913 

BOSIntell 

1 

0.10523 

0 . 12489 

0.84 

0 .4016 

BOSIntel2 

1 

-0.02569 

0 .11271 

-0.23 

0 . 8202 

BOSIntel3 

1 

-0.04930 

0 .11573 

-0.43 

0 . 6711 

BOSIntel4 

1 

-0.20733 

0 . 14197 

-1 .46 

0 . 1476 

BOSManl 

1 

0.50990 

0 .11313 

4 . 51 

<.0001 

BOSMan2 

1 

-0.13062 

0 . 10516 

-1.24 

0.2174 

BOSFS2 

1 

0 . 14203 

0 . 05784 

2 .46 

0 . 0159 

BOSFS7 

1 

-0.02651 

0.06906 

-0.38 

0 . 7020 

BOSFS9 

1 

-0 . 35710 

0.06568 

-5.44 

<.0001 

BOSMAS1 

1 

-0 . 14351 

0.06108 

-2.35 

0.0209 

BOSMAS2 

1 

0.08829 

0 .06644 

1.33 

0 . 1872 

BOSMAS4 

1 

-0.06083 

0.09128 

-0.67 

0.5068 

BOSMAS5 

1 

0 . 02722 

0.03340 

0.81 

0 . 4172 

BOSMAS6 

1 

0 . 13425 

0 . 17242 

0.78 

0 . 4382 

BOSMAS7 

1 

0 . 10711 

0 . 10276 

1 . 04 

0.3000 

BOSADA1 

1 

-0 . 42763 

0.32653 

-1.31 

0.1936 

BOSADA2 

1 

-0 . 15383 

0 . 31014 

-0.50 

0 . 6211 

BOSCC1 

1 

-0.22714 

0 .11242 

-2 . 02 

0.0462 

BOSCC2 

1 

0.06196 

0 .11371 

0 . 54 

0 . 5872 

BOSCC3 

1 

0 . 12612 

0 . 13785 

0.91 

0.3626 

BRIntel 

1 

0 . 04022 

0 .09142 

0 . 44 

0.6610 

BRMAS 

1 

0.32586 

0 .19046 

1 . 71 

0.0905 

COMAFATDS 

1 

-0 . 44792 

0.13139 

-3.41 

0.0010 

COMASAS 

1 

0 . 13750 

0 . 15381 

0.89 

0.3737 

COMMCS 

1 

0.38509 

0.13902 

2 . 77 

0.0068 

COMAMDWS 

1 

-0.25336 

0 . 13647 

-1.86 

0.0666 

INTELTUAV 

1 

0 . 00857 

0 . 07059 

0 . 12 

0.9037 

MISCBattleTiming 

1 

-0.03896 

0 . 30047 

-0.13 

0 .8971 

MISCBattleTempo 

1 

0 . 44307 

0.22766 

1 . 95 

0 . 0547 

MISCReconOps 

1 

0.09028 

0 . 05776 

1 . 56 

0 . 1215 

MISCInfoOps 

1 

-0.00563 

0.07832 

-0.07 

0 . 9428 

MISCTOCAct ivity 

1 

-0.06793 

0 . 07135 

-0.95 

0.3436 

MOPPLevel 

1 

0.36835 

0 . 74001 

0.50 

0.6198 

Staf fHud 

1 

-0.27027 

0 . 04849 

-5.57 

<.0001 

PERSCMDR 

1 

0 . 01848 

0.04032 

0.46 

0 . 6477 

PERSXO 

1 

-0 . 04533 

0 . 03755 

-1.21 

0.2305 

PERSBC 

1 

-0 . 04337 

0 . 05420 

-0.80 

0 . 4257 

PERSS3 

1 

0.03500 

0 . 05253 

0.67 

0.5068 

PERSS3NCO 

1 

0 . 02173 

0 . 05471 

0.40 

0.6922 

PERSS3RTO 

1 

0 . 17342 

0 . 05756 

3.01 

0.0033 

PERSS2 

1 

-0 .06445 

0.03616 

-1 . 78 

0.0780 

PERSFSO 

1 

0 . 00537 

0 . 04414 

0 . 12 

0.9035 

PERSENGR 

1 

0 . 05851 

0 . 05146 

1 .  14 

0.2585 

PERSALO 

1 

-0 . 07170 

0.03588 

-2.00 

0.0486 

PERSS3SGM 

1 

-0 . 07524 

0.04090 

-1 . 84 

0.0691 

Humidity 

1 

-0 . 11565 

0.10062 

-1 .  15 

0.2534 
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Regression  Analysis  for  Exercises  1,2,3 

12:04  Wednesday,  December  18, 

The  REG  Procedure 
Model:  MODEL12 
Dependent  Variable:  TA21 
Analysis  of  Variance 


Sum  of 

Mean 

Source 

DF 

Squares 

Square 

F  Value 

Pr  >  F 

Model 

46 

17.92969 

0.38978 

60.95 

<.0001 

Error 

92 

0.58830 

0.00639 

Corrected  Total 

138 

18 . 51799 

Root  MSE 

0 . 07997 

R-Square 

0.9682 

Dependent 

Mean 

0 . 15827 

Adj  R-Sq 

0.9523 

Coeff  Var 

50 . 52380 

Parameter  Estimates 

Parameter 

Standard 

Variable 

DF 

Estimate 

Error 

t  Value 

Pr  >  1 1  | 

Intercept 

1 

0.28566 

0.26752 

1 . 07 

0.2884 

BOSIntell 

1 

-0 . 30945 

0.09934 

-3.12 

0.0025 

BOSIntel2 

1 

-0.03085 

0.08965 

-0.34 

0 . 7316 

BOSIntel3 

1 

-0.03198 

0.09206 

-0.35 

0 . 7291 

BOSIntel4 

1 

-0.13186 

0 .11293 

-1 . 17 

0.2460 

BOSManl 

1 

-0.21949 

0.08998 

-2 . 44 

0.0166 

BOSMan2 

1 

0 . 11595 

0.08365 

1.39 

0.1691 

BOSFS2 

1 

0 . 08464 

0 . 04601 

1 . 84 

0.0690 

BOSFS7 

1 

-0 . 05354 

0.05493 

-0.97 

0.3323 

BOSFS9 

1 

0 . 58100 

0.05225 

11 . 12 

<.0001 

BOSMAS1 

1 

-0 . 14157 

0 . 04858 

-2 . 91 

0 . 0045 

BOSMAS2 

1 

0.08522 

0 . 05285 

1.61 

0.1103 

BOSMAS4 

1 

0.02048 

0 . 07261 

0.28 

0 . 7785 

BOSMAS5 

1 

-0.02678 

0.02656 

-1.01 

0.3161 

BOSMAS6 

1 

0.07652 

0 . 13715 

0.56 

0 . 5782 

BOSMAS7 

1 

-0.22744 

0 . 08174 

-2 . 78 

0.0065 

BOSADA1 

1 

-0.38811 

0.25973 

-1 .49 

0 . 1385 

BOSADA2 

1 

-0 . 48473 

0.24670 

-1.96 

0 . 0524 

BOSCC1 

1 

0.34808 

0.08943 

3.89 

0.0002 

BOSCC2 

1 

0.19068 

0 .09045 

2 . 11 

0 . 0377 

BOSCC3 

1 

0.27962 

0 . 10965 

2 . 55 

0 . 0124 

BRIntel 

1 

-0.03842 

0 . 07272 

-0.53 

0.5986 

BRMAS 

1 

-0 . 02476 

0 . 15150 

-0.16 

0 . 8705 

COMAFATDS 

1 

-0.21779 

0 . 10452 

-2.08 

0.0400 

COMASAS 

1 

0.09225 

0 . 12235 

0 . 75 

0 . 4528 

COMMCS 

1 

0 . 18121 

0 .11058 

1 . 64 

0 . 1047 

COMAMDWS 

1 

-0 . 14486 

0 . 10856 

-1.33 

0 . 1853 

INTELTUAV 

1 

-0 . 05897 

0 . 05615 

-1 . 05 

0.2964 

MISCBattleTiming 

1 

-0.73666 

0.23900 

-3.08 

0 . 0027 

MISCBattleTempo 

1 

0.38689 

0.18109 

2 . 14 

0.0353 

MISCReconOps 

1 

0 .06274 

0 . 04594 

1 . 37 

0 . 1754 

MISCInfoOps 

1 

0.02978 

0.06230 

0.48 

0.6338 

MISCTOCAct ivity 

1 

-0.12838 

0 . 05676 

-2.26 

0.0261 

MOPPLevel 

1 

-0.66156 

0.58863 

-1 .  12 

0.2640 

Staf fHud 

1 

-0 . 15488 

0 . 03857 

-4 . 02 

0.0001 

PERSCMDR 

1 

0 .01156 

0.03207 

0.36 

0 . 7192 

PERSXO 

1 

-0 . 00957 

0.02987 

-0.32 

0 . 7494 

PERSBC 

1 

-0 . 04375 

0 . 04311 

-1.01 

0.3129 

PERSS3 

1 

-0 . 02773 

0 . 04178 

-0.66 

0.5086 

PERSS3NCO 

1 

0 . 04314 

0.04352 

0.99 

0 . 3241 

PERSS3RTO 

1 

0.23373 

0 . 04579 

5.10 

<.0001 

PERSS2 

1 

-0 . 01221 

0.02876 

-0 . 42 

0 . 6721 

PERSFSO 

1 

-0 . 00140 

0 . 03511 

-0.04 

0.9683 

PERSENGR 

1 

0 . 05674 

0.04093 

1.39 

0.1690 

PERSALO 

1 

-0.06225 

0 . 02854 

-2 . 18 

0 . 0317 

PERSS3SGM 

1 

-0.05630 

0.03254 

-1 . 73 

0.0869 

Humidity 

1 

0 . 14449 

0.08004 

1.81 

0 . 0743 

23 
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Regression  Analysis  for  Exercises  1,2,3 

12:04  Wednesday,  December  18, 

The  REG  Procedure 
Model:  MODEL13 
Dependent  Variable:  TA23 
Analysis  of  Variance 


Sum  of 

Mean 

Source 

DF 

Squares 

Square 

F  Value 

Pr  >  F 

Model 

46 

20 . 61156 

0.44808 

36.04 

<.0001 

Error 

92 

1 . 14383 

0.01243 

Corrected  Total 

138 

21 . 75540 

Root  MSE 

0 .11150 

R-Square 

0 . 9474 

Dependent 

Mean 

0 .19424 

Adj  R-Sq 

0 . 9211 

Coeff  Var 

57 . 40347 

Parameter  Estimates 

Parameter 

Standard 

Variable 

DF 

Estimate 

Error 

t  Value 

Pr  >  | t | 

Intercept 

1 

-1 . 10940 

0.37302 

-2 . 97 

0.0038 

BOSIntell 

1 

0.62619 

0.13852 

4 . 52 

<.0001 

BOSIntel2 

1 

0.63168 

0 . 12501 

5.05 

<.0001 

BOSIntel3 

1 

0 . 64790 

0.12836 

5.05 

<.0001 

BOSIntel4 

1 

0 .81110 

0 . 15747 

5.15 

<.0001 

BOSManl 

1 

0.00912 

0 . 12547 

0.07 

0 . 9422 

BOSMan2 

1 

0.03763 

0 .11664 

0.32 

0 . 7477 

BOSFS2 

1 

0.02030 

0 .06415 

0.32 

0 . 7523 

BOSFS7 

1 

0 . 04589 

0 . 07659 

0.60 

0 . 5505 

BOSFS9 

1 

0.24031 

0 . 07285 

3.30 

0 . 0014 

BOSMAS1 

1 

-0 . 13789 

0 .06774 

-2 . 04 

0 . 0447 

BOSMAS2 

1 

0 . 07226 

0.07369 

0.98 

0.3293 

BOSMAS4 

1 

0 .09674 

0 . 10125 

0.96 

0 . 3418 

BOSMAS5 

1 

0.03305 

0 . 03704 

0.89 

0.3746 

BOSMAS6 

1 

0 .76104 

0 .19124 

3.98 

0.0001 

BOSMAS7 

1 

0 . 01922 

0.11398 

0 . 17 

0.8665 

BOSADA1 

1 

-0.92989 

0 . 36217 

-2 . 57 

0.0119 

BOSADA2 

1 

-0 . 85743 

0.34399 

-2 .49 

0 . 0145 

BOSCC1 

1 

0.52060 

0 . 12469 

4 . 17 

<.0001 

BOSCC2 

1 

0.39915 

0 . 12612 

3.16 

0.0021 

BOSCC3 

1 

0 . 36755 

0.15289 

2 . 40 

0.0182 

BRIntel 

1 

-0 . 51395 

0 . 10140 

-5.07 

<.0001 

BRMAS 

1 

0 . 13242 

0.21125 

0.63 

0.5323 

COMAFATDS 

1 

0 . 12878 

0 . 14573 

0.88 

0 . 3792 

COMASAS 

1 

0.16697 

0.17060 

0.98 

0.3303 

COMMCS 

1 

0 . 05789 

0 . 15420 

0.38 

0 . 7082 

COMAMDWS 

1 

-0.12696 

0 . 15137 

-0.84 

0.4038 

INTELTUAV 

1 

-0 .06047 

0.07829 

-0 . 77 

0 .4419 

MISCBattleTiming 

1 

0 . 84542 

0.33326 

2 . 54 

0.0129 

MISCBattleTempo 

1 

-0.39560 

0.25251 

-1 . 57 

0.1206 

MISCReconOps 

1 

0 . 01579 

0.06406 

0.25 

0.8059 

MISCInfoOps 

1 

-0 . 12614 

0.08687 

-1 . 45 

0 . 1499 

MISCTOCAct ivity 

1 

0 . 14106 

0 . 07914 

1 . 78 

0.0780 

MOPPLevel 

1 

2 . 15706 

0.82078 

2.63 

0.0101 

Staf fHud 

1 

0.20482 

0 . 05378 

3.81 

0.0003 

PERSCMDR 

1 

-0.08859 

0 . 04472 

-1.98 

0.0506 

PERSXO 

1 

-0.06267 

0 . 04165 

-1 . 50 

0 . 1358 

PERSBC 

1 

0 . 14091 

0.06012 

2 . 34 

0 . 0212 

PERSS3 

1 

0 . 12729 

0.05826 

2 . 18 

0 . 0315 

PERSS3NCO 

1 

-0 . 10255 

0.06068 

-1.69 

0 .0944 

PERSS3RTO 

1 

-0.13052 

0.06384 

-2 . 04 

0.0438 

PERSS2 

1 

-0 . 00575 

0 . 04011 

-0 . 14 

0.8863 

PERSFSO 

1 

-0 . 05851 

0.04896 

-1.20 

0.2351 

PERSENGR 

1 

-0.03081 

0 . 05708 

-0 . 54 

0.5906 

PERSALO 

1 

0.03630 

0.03980 

0.91 

0.3641 

PERSS3SGM 

1 

-0 . 01555 

0 . 04537 

-0.34 

0 . 7325 

Humidity 

1 

-0 . 18498 

0 .11161 

-1.66 

0.1008 
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Regression  Analysis  for  Exercises  1,2,3 

12:04  Wednesday,  December  18, 

The  REG  Procedure 
Model:  MODEL14 

Dependent  Variable:  Missionl 
Analysis  of  Variance 


Sum  of 

Mean 

Source 

DF 

Squares 

Square 

F  Value 

Pr  >  F 

Model 

46 

13.74863 

0.29888 

18.68 

<.0001 

Error 

92 

1 . 47209 

0.01600 

Corrected  Total 

138 

15.22072 

Root  MSE 

0.12650 

R-Square 

0.9033 

Dependent 

Mean 

0 .16835 

Adj  R-Sq 

0 . 8549 

Coeff  Var 

75.14023 

Parameter  Estimates 

Parameter 

Standard 

Variable 

DF 

Estimate 

Error 

t  Value 

Pr  >  1 1  | 

Intercept 

1 

0 . 58670 

0 . 42318 

1.39 

0.1690 

BOSIntell 

1 

-0 . 14253 

0 . 15715 

-0.91 

0.3668 

BOSIntel2 

1 

0.30878 

0 . 14182 

2 . 18 

0.0320 

BOSIntel3 

1 

0.29769 

0 . 14562 

2 . 04 

0.0438 

BOSIntel4 

1 

0.09400 

0 . 17864 

0.53 

0.6000 

BOSManl 

1 

0.03386 

0 . 14234 

0.24 

0.8125 

BOSMan2 

1 

0.20085 

0.13233 

1.52 

0.1325 

BOSFS2 

1 

0 . 13850 

0 . 07277 

1.90 

0.0601 

BOSFS7 

1 

-0 . 01456 

0.08689 

-0 . 17 

0.8673 

BOSFS9 

1 

0 . 07701 

0.08265 

0.93 

0.3539 

BOSMAS1 

1 

-0.25223 

0 . 07685 

-3.28 

0 . 0015 

BOSMAS2 

1 

0 . 31780 

0.08360 

3.80 

0.0003 

BOSMAS4 

1 

0.33284 

0 .11486 

2.90 

0 . 0047 

BOSMAS5 

1 

-0 . 01144 

0 . 04202 

-0.27 

0.7860 

BOSMAS6 

1 

0.21109 

0.21696 

0.97 

0.3331 

BOSMAS7 

1 

-0.23476 

0.12930 

-1 . 82 

0 . 0727 

BOSADA1 

1 

-1 . 04018 

0 .41087 

-2.53 

0.0130 

BOSADA2 

1 

-0.70643 

0.39024 

-1.81 

0 . 0735 

BOSCC1 

1 

0.01598 

0 . 14146 

0.11 

0.9103 

BOSCC2 

1 

0 . 00052177 

0 . 14307 

0.00 

0 . 9971 

BOSCC3 

1 

-0.33436 

0 . 17345 

-1.93 

0 . 0570 

BRIntel 

1 

-0 . 15512 

0 .11503 

-1 . 35 

0.1808 

BRMAS 

1 

0.06856 

0.23965 

0.29 

0 . 7754 

COMAFATDS 

1 

0.06891 

0 .16533 

0 . 42 

0 . 6778 

COMASAS 

1 

-0 .06757 

0.19353 

-0.35 

0 . 7278 

COMMCS 

1 

0 . 38756 

0 . 17493 

2.22 

0 . 0292 

COMAMDWS 

1 

-0 . 00114 

0 . 17172 

-0.01 

0 . 9947 

INTELTUAV 

1 

0 . 02355 

0.08882 

0.27 

0 .7915 

MISCBattleTiming 

1 

-1 . 33485 

0 . 37807 

-3.53 

0.0006 

MISCBattleTempo 

1 

0 . 15631 

0.28646 

0 . 55 

0.5866 

MISCReconOps 

1 

0.08690 

0 . 07267 

1.20 

0.2349 

MISCInfoOps 

1 

-0 . 07420 

0 .09855 

-0 . 75 

0 .4534 

MISCTOCAct ivity 

1 

-0 . 02411 

0.08978 

-0.27 

0.7889 

MOPPLevel 

1 

1 .79388 

0 . 93114 

1.93 

0 . 0571 

Staf fHud 

1 

-0 . 05254 

0.06101 

-0.86 

0.3913 

PERSCMDR 

1 

-0.03789 

0 . 05073 

-0 . 75 

0 . 4571 

PERSXO 

1 

-0.03168 

0 . 04725 

-0.67 

0 . 5042 

PERSBC 

1 

-0 . 03527 

0.06820 

-0.52 

0.6063 

PERSS3 

1 

0.00163 

0.06610 

0.02 

0.9803 

PERSS3NCO 

1 

-0 . 01232 

0.06884 

-0.18 

0.8583 

PERSS3RTO 

1 

0 . 14674 

0 . 07243 

2.03 

0 . 0457 

PERSS2 

1 

-0 . 04526 

0 . 04550 

CTl 

O'! 

o 

1 

0.3224 

PERSFSO 

1 

0 . 04818 

0 . 05554 

0.87 

0.3879 

PERSENGR 

1 

-0 . 05707 

0 .06475 

-0.88 

0.3804 

PERSALO 

1 

-0.05026 

0 . 04515 

-1 . 11 

0.2685 

PERSS3SGM 

1 

-0.13839 

0 . 05147 

-2.69 

0.0085 

Humidity 

1 

-0.07062 

0.12661 

-0.56 

0 . 5783 
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Regression  Analysis  for  Exercises  1,2,3 

12:04  Wednesday,  December  18, 

The  REG  Procedure 
Model:  MODEL15 

Dependent  Variable:  Mission2 
Analysis  of  Variance 


Sum  of 

Mean 

Source 

DF 

Squares 

Square 

F  Value 

Pr  >  F 

Model 

46 

5.48486 

0 . 11924 

19.93 

<.0001 

Error 

92 

0 . 55039 

0.00598 

Corrected  Total 

138 

6 . 03525 

Root  MSE 

0 . 07735 

R-Square 

0.9088 

Dependent 

Mean 

0.22158 

Adj  R-Sq 

0.8632 

Coeff  Var 

34.90639 

Parameter  Estimates 

Parameter 

Standard 

Variable 

DF 

Estimate 

Error 

t  Value 

Pr  >  1 1  | 

Intercept 

1 

0.60367 

0.25876 

2.33 

0.0218 

BOSIntell 

1 

0 . 15894 

0.09609 

1 . 65 

0 . 1015 

BOSIntel2 

1 

-0 . 02770 

0.08671 

-0.32 

0 . 7502 

BOSIntel3 

1 

0.00030605 

0.08904 

0.00 

0.9973 

BOSIntel4 

1 

-0 .09494 

0.10923 

-0.87 

0.3870 

BOSManl 

1 

0 . 02077 

0 . 08704 

0.24 

0.8119 

BOSMan2 

1 

-0 . 08164 

0.08091 

-1.01 

0 . 3156 

BOSFS2 

1 

-0.20260 

0 . 04450 

-4 . 55 

<.0001 

BOSFS7 

1 

0.09261 

0 . 05313 

1 . 74 

0.0846 

BOSFS9 

1 

-0.07088 

0 . 05053 

-1 . 40 

0 .1641 

BOSMAS1 

1 

0.26895 

0.04699 

5.72 

<.0001 

BOSMAS2 

1 

-0 . 00567 

0 . 05112 

-0.11 

0.9120 

BOSMAS4 

1 

0 .11842 

0.07023 

1.69 

0.0952 

BOSMAS5 

1 

0 . 01779 

0.02569 

0.69 

0 .4904 

BOSMAS6 

1 

-0.03803 

0.13266 

-0.29 

0 . 7750 

BOSMAS7 

1 

-0.02946 

0.07906 

-0.37 

0 . 7103 

BOSADA1 

1 

-0.27811 

0.25123 

-1 . 11 

0.2712 

BOSADA2 

1 

-0 . 47268 

0.23862 

-1.98 

0.0506 

BOSCC1 

1 

0.05383 

0.08650 

0.62 

0 . 5353 

BOSCC2 

1 

0.23177 

0 . 08748 

2.65 

0.0095 

BOSCC3 

1 

0 . 31873 

0.10606 

3.01 

0.0034 

BRIntel 

1 

-0 . 14474 

0 . 07034 

-2.06 

0 . 0424 

BRMAS 

1 

-0.10263 

0 . 14654 

-0.70 

0 . 4854 

COMAFATDS 

1 

0.32286 

0.10109 

3.19 

0.0019 

COMASAS 

1 

-0 . 17358 

0 .11834 

-1 . 47 

0 . 1458 

COMMCS 

1 

0 . 10561 

0.10696 

0.99 

0.3261 

COMAMDWS 

1 

-0 . 17858 

0 . 10500 

-1 . 70 

0 .0924 

INTELTUAV 

1 

0.03406 

0 . 05431 

0.63 

0.5321 

MISCBattleTiming 

1 

-0.39661 

0.23118 

-1 . 72 

0.0896 

MISCBattleTempo 

1 

0 . 15097 

0 . 17516 

0.86 

0.3910 

MISCReconOps 

1 

-0 .09537 

0 . 04444 

-2.15 

0 . 0345 

MISCInfoOps 

1 

0.06634 

0.06026 

1.10 

0.2738 

MISCTOCAct ivity 

1 

0.01980 

0 . 05490 

0.36 

0 . 7191 

MOPPLevel 

1 

-0.21559 

0 . 56935 

-0.38 

0 . 7058 

Staf fHud 

1 

0.09636 

0.03730 

2.58 

0 . 0114 

PERSCMDR 

1 

0.03327 

0.03102 

1 . 07 

0.2863 

PERSXO 

1 

-0.03878 

0.02889 

-1 . 34 

0 . 1827 

PERSBC 

1 

0.03041 

0 . 04170 

0.73 

0 .4677 

PERSS3 

1 

0 .06412 

0 . 04041 

1 . 59 

0.1160 

PERSS3NCO 

1 

-0.11083 

0.04209 

-2.63 

0.0099 

PERSS3RTO 

1 

0 . 00721 

0 . 04429 

0.16 

0 . 8710 

PERSS2 

1 

0 . 03407 

0 . 02782 

1.22 

0.2238 

PERSFSO 

1 

-0 . 00771 

0.03396 

-0.23 

0.8210 

PERSENGR 

1 

-0 . 02104 

0.03959 

-0.53 

0 .5965 

PERSALO 

1 

-0 . 01419 

0 . 02761 

-0 . 51 

0.6086 

PERSS3SGM 

1 

0.05189 

0 . 03147 

1 . 65 

0.1026 

Humidity 

1 

-0 . 05031 

0 . 07742 

-0.65 

0 . 5174 
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Regression  Analysis  for  Exercises  1,2,3 

12:04  Wednesday,  December  18, 

The  REG  Procedure 
Model:  MODEL16 
Dependent  Variable:  Mission3 
Analysis  of  Variance 

Sum  of  Mean 


Source 

DF 

Squares 

Square 

F  Value 

Pr  >  ] 

Model 

46 

5.69368 

0 . 12378 

9.64 

<.000 

Error 

92 

1 . 18150 

0 . 01284 

Corrected  Total 

138 

6.87518 

Root  MSE 

0 . 11332 

R-Square 

0.8281 

Dependent 

Mean 

0 . 08417 

Adj  R-Sq 

0 . 7422 

Coeff  Var 

134.63326 

Parameter  Estimates 

Parameter 

Standard 

Variable 

DF 

Estimate 

Error 

t  Value 

Pr  >  | t | 

Intercept 

1 

-1 . 71231 

0 . 37912 

-4 . 52 

<.0001 

BOSIntell 

1 

-0 .01427 

0 . 14078 

-0.10 

0 . 9195 

BOSIntel2 

1 

0.24020 

0 . 12705 

1.89 

0.0618 

BOSIntel3 

1 

0.22497 

0.13046 

1 . 72 

0.0880 

BOSIntel4 

1 

0.39789 

0 .16004 

2 .49 

0 . 0147 

BOSManl 

1 

-0.22547 

0 . 12752 

-1 . 77 

0.0804 

BOSMan2 

1 

0 . 10505 

0 .11855 

0.89 

0 . 3778 

BOSFS2 

1 

0.00856 

0.06520 

0.13 

0.8959 

BOSFS7 

1 

-0 . 15640 

0 . 07784 

-2 . 01 

0 . 0475 

BOSFS9 

1 

-0 . 05179 

0 . 07404 

-0.70 

0.4860 

BOSMAS1 

1 

-0 . 04242 

0.06885 

-0.62 

0.5393 

BOSMAS2 

1 

0 . 07679 

0 . 07489 

1.03 

0.3079 

BOSMAS4 

1 

-0 . 35597 

0.10290 

-3.46 

0.0008 

BOSMAS5 

1 

-0.03636 

0 . 03765 

-0.97 

0.3366 

BOSMAS6 

1 

0.10902 

0 .19437 

0.56 

0.5762 

BOSMAS7 

1 

0 . 15132 

0 .11584 

1.31 

0 .1947 

BOSADA1 

1 

-0.23917 

0.36809 

-0.65 

0 . 5175 

BOSADA2 

1 

-0.00037762 

0.34961 

-0.00 

0.9991 

BOSCC1 

1 

0.23068 

0 . 12673 

1 . 82 

0 . 0720 

BOSCC2 

1 

-0.00368 

0 . 12818 

-0.03 

0 . 9772 

BOSCC3 

1 

0 . 15248 

0 . 15539 

0.98 

0.3290 

BRIntel 

1 

0 .19416 

0.10306 

1.88 

0.0627 

BRMAS 

1 

0 . 52344 

0.21470 

2 . 44 

0.0167 

COMAFATDS 

1 

0 . 07444 

0 . 14811 

0.50 

0 . 6165 

COMASAS 

1 

-0.27208 

0 . 17338 

-1 . 57 

0.1200 

COMMCS 

1 

-0.39090 

0 . 15672 

-2 .49 

0 . 0144 

COMAMDWS 

1 

0.24650 

0 . 15384 

1.60 

0 . 1125 

INTELTUAV 

1 

-0 .19124 

0 . 07957 

-2 . 40 

0.0183 

MISCBattleTiming 

1 

1 .09595 

0 . 33871 

3.24 

0 . 0017 

MISCBattleTempo 

1 

-0.90338 

0.25664 

-3.52 

0.0007 

MISCReconOps 

1 

-0.02601 

0 .06511 

-0.40 

0.6904 

MISCInfoOps 

1 

0 . 14085 

0.08829 

1.60 

0 . 1141 

MISCTOCAct ivity 

1 

-0 . 10201 

0.08043 

-1.27 

0.2079 

MOPPLevel 

1 

1.86234 

0.83419 

2.23 

0.0280 

Staf fHud 

1 

-0.04033 

0 . 05466 

-0 . 74 

0 .4625 

PERSCMDR 

1 

-0.09343 

0 . 04545 

-2.06 

0.0426 

PERSXO 

1 

0 .09437 

0.04233 

2.23 

0 . 0282 

PERSBC 

1 

0.10010 

0.06110 

1 . 64 

0 . 1048 

PERSS3 

1 

-0.02298 

0.05921 

-0.39 

0.6989 

PERSS3NCO 

1 

0.08349 

0.06167 

1 . 35 

0 . 1791 

PERSS3RTO 

1 

0 . 12537 

0.06489 

1.93 

0 . 0564 

PERSS2 

1 

0 . 04078 

0 . 04076 

1.00 

0 . 3197 

PERSFSO 

1 

-0 . 04456 

0 . 04976 

-0.90 

0 . 3728 

PERSENGR 

1 

0 . 10094 

0 . 05801 

1 . 74 

0.0852 

PERSALO 

1 

-0.06638 

0 . 04045 

-1 . 64 

0 . 1042 

PERSS3SGM 

1 

0.03336 

0 . 04611 

0 . 72 

0 . 4712 

Humidity 

1 

0.42989 

0 .11343 

3.79 

0.0003 
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Regression  Analysis  for  Exercises  1,2,3 

12:04  Wednesday,  December  18, 

The  REG  Procedure 
Model:  MODEL17 

Dependent  Variable:  Mission5 
Analysis  of  Variance 


Sum  of 

Mean 

Source 

DF 

Squares 

Square 

F  Value 

Pr  >  F 

Model 

46 

5.21520 

0 . 11337 

8 . 74 

<.0001 

Error 

92 

1.19386 

0.01298 

Corrected  Total 

138 

6.40906 

Root  MSE 

0 . 11392 

R-Square 

0 . 8137 

Dependent 

Mean 

0 . 07050 

Adj  R-Sq 

0.7206 

Coeff  Var 

161 . 57404 

Parameter  Estimates 

Parameter 

Standard 

Variable 

DF 

Estimate 

Error 

t  Value 

Pr  >  1 1  | 

Intercept 

1 

-0.32019 

0.38110 

-0.84 

0.4030 

BOSIntell 

1 

-0 .16441 

0 . 14152 

-1.16 

0.2483 

BOSIntel2 

1 

-0.22837 

0 . 12771 

-1 .79 

0 . 0770 

BOSIntel3 

1 

-0.27342 

0 . 13114 

-2.08 

0.0398 

BOSIntel4 

1 

-0 . 35546 

0.16088 

-2.21 

0.0296 

BOSManl 

1 

0.66637 

0 . 12819 

5.20 

<.0001 

BOSMan2 

1 

0 . 10417 

0 .11917 

0.87 

0.3843 

BOSFS2 

1 

0 . 07137 

0 .06554 

1.09 

0.2790 

BOSFS7 

1 

0.09092 

0 . 07825 

1.16 

0.2483 

BOSFS9 

1 

-0.22286 

0 . 07443 

-2.99 

0.0035 

BOSMAS1 

1 

0.09643 

0.06921 

1.39 

0.1669 

BOSMAS2 

1 

-0.10678 

0 . 07528 

-1 . 42 

0 . 1595 

BOSMAS4 

1 

-0.09509 

0 . 10344 

-0.92 

0.3604 

BOSMAS5 

1 

0 . 02075 

0 . 03784 

0 . 55 

0 . 5848 

BOSMAS6 

1 

-0.35216 

0.19538 

-1.80 

0 . 0747 

BOSMAS7 

1 

0.01205 

0 .11644 

0.10 

0 . 9178 

BOSADA1 

1 

0 . 67241 

0 . 37001 

1 . 82 

0 . 0724 

BOSADA2 

1 

0 . 97435 

0 . 35143 

2 . 77 

0.0067 

BOSCC1 

1 

-0 . 14022 

0 . 12739 

-1.10 

0.2739 

BOSCC2 

1 

-0.08028 

0 . 12884 

-0.62 

0 . 5348 

BOSCC3 

1 

0 . 13358 

0.15620 

0.86 

0 .3947 

BRIntel 

1 

0 . 07793 

0.10359 

0 . 75 

0 . 4538 

BRMAS 

1 

0.10393 

0.21582 

0.48 

0.6313 

COMAFATDS 

1 

-0 . 47430 

0 . 14889 

-3.19 

0.0020 

COMASAS 

1 

0.38918 

0 . 17429 

2.23 

0.0280 

COMMCS 

1 

-0.21810 

0 . 15753 

-1.38 

0.1696 

COMAMDWS 

1 

0.20595 

0 . 15464 

1.33 

0.1862 

INTELTUAV 

1 

0.00646 

0.07999 

0.08 

0.9358 

MISCBattleTiming 

1 

0 . 88944 

0 . 34047 

2.61 

0 . 0105 

MISCBattleTempo 

1 

0.36489 

0.25797 

1 . 41 

0.1606 

MISCReconOps 

1 

0 . 07773 

0 .06544 

1.19 

0.2380 

MISCInfoOps 

1 

-0.03088 

0 . 08875 

-0.35 

0 . 7287 

MISCTOCAct ivity 

1 

0 . 00641 

0.08085 

0.08 

0.9370 

MOPPLevel 

1 

-2 . 15733 

0 . 83854 

-2 . 57 

0 . 0117 

Staf fHud 

1 

-0.03290 

0 . 05494 

-0.60 

0 . 5508 

PERSCMDR 

1 

0.01899 

0 . 04569 

0 . 42 

0 . 6787 

PERSXO 

1 

-0 .06518 

0 . 04255 

-1 . 53 

0.1290 

PERSBC 

1 

-0 . 04891 

0 .06142 

-0.80 

0 . 4279 

PERSS3 

1 

-0.06338 

0 . 05952 

-1.06 

0.2897 

PERSS3NCO 

1 

0 . 12831 

0.06199 

2 . 07 

0 . 0413 

PERSS3RTO 

1 

-0.06936 

0.06522 

-1.06 

0.2904 

PERSS2 

1 

-0 .06051 

0 . 04097 

-1 . 48 

0 . 1431 

PERSFSO 

1 

0.03079 

0.05002 

0.62 

0 . 5397 

PERSENGR 

1 

-0.09029 

0 . 05831 

-1 . 55 

0.1250 

PERSALO 

1 

0.13935 

0.04066 

3.43 

0.0009 

PERSS3SGM 

1 

0 . 05592 

0 . 04635 

1.21 

0.2308 

Humidity 

1 

-0.00103 

0 .11402 

-0.01 

0.9928 

33 

2002 


403 


Appendix  K  -  Second  Linear  Regression  Run  To  Select  IV  For  Inclusion  In  DV  Models 


SAS  Code  (Pick  those  IV  that  had  a  significant p  value  (a  =  .05)  from  the  first  run): 

options  pageno=l  f ormdlim="-" ; 

TITLE  "Training  Data  123  Regression  Analysis'; 

DATA  Tngdatal23; 

INPUT  Dayl  Day2  Day3  Day4  Day5  Day6  Day7  Day8  Day9  DaylO  Dayll  Dayl2  Dayl3  Dayl4  Dayl5  Dayl6 
Dayl7  Dayl8  Dayl9  Day20  Hourl  Hour2  Hour3  Hour4  Hour5  Hour6  Hour7  Hour8  Hour9  HourlO  Hourll 
Hourl2  Hourl3  Hourl4  Hourl5  Hourl6  Hourl7  Hourl8  Hourl9  Hour20  Hour21  Hour22  Hour23  Hour24 
TOCAn  TOCDg  TOCBn  TOCBde  ExTypl  ExTyp2  ExTyp3  ExTyp4  ExNol  ExNo2  ExNo3  ExNo4  ExNo5  ExNo6  ExNo7 
ExNo8  ExNo9  ExNolO  ExNoll  ExNol2  ExNol3  ExNol4  ExNol5  ExNol6  ExNol7  ExNol8  ExNol9  ExNo20  SimNo 
SimJANS  SimJAST  SimCBS  SimCCTT  SimSTRM  BOSIntell  BOSIntel2  BOSIntel3  BOSIntel4  BOSManl  BOSMan2 
BOSFS1  BOSFS2  BOSFS3  BOSFS4  BOSFS5  BOSFS6  BOSFS7  BOSFS8  BOSFS9  BOSMAS1  BOSMAS2  BOSMAS3  BOSMAS4 
BOSMAS5  BOSMAS6  BOSMAS7  BOSADA1  BOSADA2  BOSCC1  BOSCC2  BOSCC3  BRIntel  BRMan  BRFS  BRMAS  BRADA 
BRCSS  BRC2  COMAFATDS  COMAMPS  COMASAS  COMCTIS  COMCSSCS  COMFAADC2  COMFBCB2  COMIMETS  COMMCS  COMAMDWS 
COMJSTARS  COMEPLARS  INTELTUAV  INTELJSTARS  INTELGSR  MISCBattleTiming  MISCBattleTempo 
MISCReconOps  MISCInfoOps  MISCTactics  MISCTOCActivity  MOPPLevel  StaffHud  PERSCMDR  PERSXO  PERSBC 
PERSS3  PERSS3NCO  PERSS3RTO  PERSS2  PERSFSO  PERSENGR  PERSALO  PERSS3SGM  SKYCLR  SKYOC  SKYRAIN 
WINDNONE  WINDLOW  WINDMOD  WINDHIGH  LightBG  LightFG  Noise  TEMPDB  TEMPWB  Humidity  TP1  TP2  TP3  TP4 
TP 5  TP 6  TP 7  TP 8  TP 9  TP10  TP11  TP12  TP13  TP14  TP15  TP16  TP17  TP18  TP19  TP20  TP21  TP22  TP23  TS1 
TS2  TS3  TS4  TS5  TS6  TS7  TS8  TS9  TS10  TS11  TS12  TS13  TS14  TS15  TS16  TS17  TS18  TS19  TS20  TS21 
TS22  TS23  TA1  TA2  TA3  TA4  TA5  TA6  TA7  TA8  TA9  TA10  TA11  TA12  TA13  TA14  TA15  TA16  TA17  TA18 
TA19  TA20  TA21  TA22  TA23  Missionl  Mission2  Mission3  Mission4  Mission5  Mission6 


CARDS; 
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.3 

0. 

.  5 

0. 

.2 

0. 

.00 

0 

0 

0 

0 

0 

0  1 

10  1 

1 

0  1 

0  0 

10  0 

0 

0. 

04 

0. 

.16 

0. 

66 

0. 

.  90 

0. 

75 

0. 

.50 

0 

0  0  0 

0 

0 

0 

0 

0 

0 

1 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0  0 

0  0 

0  0 

0  0 

0  0 

0  0 

0 

0 

1 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0  0 

0  0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

1 

0 

0 

0.0 

0.4 

0.0 

0.0 

0.0 

0.0 

*  Make  regression  run  with  input/independent  variables  remaining  that  are  significant  after  full 
regression  run 

*  to  try  to  maximize  the  adjusted  r**2  showing  the  amount  of  the  regression  accounted  for; 
options  pageno=l  f ormdlim="-" ; 

proc  reg  data=Tngdatal23; 

TITLE  'second  level  Regression  Analysis  for  Exercises  1,2,3"; 

model  TP10=BOSIntel3  BOSIntel4  BOSMAS2  BOSMAS6  BOSADA1  BOSADA2  BOSCC1  BOSCC2  BOSCC3  MOPPLevel; 
model  TPll=BOSIntel3  BOSFS2  BOSFS9  BOSMAS1  BOSMAS2  BOSCC1  BOSCC2  BOSCC3  BRIntel  StaffHud  PERSXO 
PERSS3RTO  PERSS3SGM; 

model  TP12=BOSFS2  BOSFS7  BOSFS9  BOSMAS1  BOSMAS4  BOSMAS6  BOSCC1  COMAFATDS  COMASAS  COMAMDWS 
MISCBattleTiming  MISCBattleTempo 

MISCReconOps  PERSXO  PERSBC  PERSALO  Humidity; 

model  TP15=BOSIntel2  BOSIntel3  BOSIntel4  BOSManl  BOSFS2  BOSMAS2  COMAFATDS  MISCBattleTiming 
MISCReconOps  PERSXO  ; 

model  TP21=BOSFS9  BOSMAS5  MISCBattleTempo  MISCInfoOps  StaffHud  Humidity; 
model  TP2 3=BOSMAS 6  BRIntel  BRMAS  COMAMDWS  StaffHud; 
model  TS12=BOSManl  BOSFS9  BOSMAS1; 

model  TS18=BOSFS2  BOSMAS6  BOSCC1  COMAFATDS  MISCBattleTiming  MISCReconOps  MISCInfoOps  PERSXO  ; 
model  TS19=BOSMAS6  PERSS2 ; 

model  TS23=BOSIntel2  BOSIntel3  BOSMan2  BOSFS9  BOSMAS4  BOSCC2  MISCBattleTiming  MISCInfoOps 
PERSS3RTO  Humidity; 

model  TA3=BOSManl  BOSFS2  BOSFS9  BOSMAS1  BOSCC1  COMAFATDS  COMMCS  StaffHud  PERSS3RTO  PERSALO; 
model  TA21=BOSIntell  BOSManl  BOSFS9  BOSMAS1  BOSMAS7  BOSCC1  BOSCC3  COMAFATDS  MISCBattleTiming 
MISCBattleTempo 
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MISCTOCActivity  StaffHud  PERSS3RTO  PERSALO; 

model  TA23=BOSIntell  BOSIntel2  BOSIntel3  BOSIntel4  BOSFS9  BOSMAS1  BOSMAS6  BOSADA1  BOSADA2 
BOSCC1  BOSCC2  BOSCC3 

BRIntel  BRMAS  MISCBattleTiming  MOPPLevel  StaffHud  PERSBC  PERSS3  PERSS3RTO  Humidity; 
model  Missionl=BOSIntel2  BOSIntel3  BOSMAS1  BOSMAS2  BOSMAS4  BOSADA1  COMMCS  MISCBattleTiming 
PERSS3RTO  PERSS3SGM  ; 

model  Mission2=BOSFS2  BOSMAS1  BOSCC2  BOSCC3  BRIntel  COMAFATDS  MISCReconOps  StaffHud  PERSS3NCO  ; 
model  Mission3=BOSIntel4  BOSFS7  BOSMAS4  BRMAS  COMMCS  INTELTUAV  MISCBattleTiming  MISCBattleTempo 
MOPPLevel  PERSCMDR  PERSXO  Humidity; 

model  Mission5=BOSIntel3  BOSIntel4  BOSManl  BOSFS9  BOSADA2  COMAFATDS  COMASAS  MISCBattleTiming 
MOP 
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SAS  Output: 


Second  level  Regression  Analysis  for  Exercises  1,2,3  51 

12:04  Wednesday,  December  18,  2002 

The  REG  Procedure 
Model:  M0DEL1 
Dependent  Variable:  TP10 
Analysis  of  Variance 


Sum  of 

Mean 

Source 

DF 

Squares 

Square 

F  Value 

Pr  >  1 

Model 

10 

7.97293 

0.79729 

18.87 

<.000 

Error 

128 

5.40837 

0.04225 

Corrected  Total 

138 

13.38129 

Root  MSE 

0.20556 

R-Square 

0.5958 

Dependent 

Mean 

0.10791 

Adj  R-Sq 

0.5643 

Coeff  Var 

190.48100 

Parameter  Estimates 

Parameter 

Standard 

Variable 

DF 

Estimate 

Error 

t  Value 

Pr  >  | t | 

Intercept 

1 

0.44886 

0.17174 

2.61 

0.0100 

B0SIntel3 

1 

0.00466 

0.05049 

0.09 

0.9266 

B0SIntel4 

1 

0.27463 

0.08769 

3.13 

0.0022 

B0SMAS2 

1 

0.25440 

0.06685 

3.81 

0.0002 

B0SMAS6 

1 

-0.21173 

0.14555 

-1.45 

0.1482 

B0SADA1 

1 

0.22378 

0.16293 

1  .37 

0.1720 

B0SADA2 

1 

-0.01263 

0.12326 

-0.10 

0.9186 

B0SCC1 

1 

-0.22450 

0.14833 

-1.51 

0.1326 

B0SCC2 

1 

-0.53501 

0.16576 

-3.23 

0.0016 

B0SCC3 

1 

-0.34403 

0.17206 

-2.00 

0.0477 

MOPPLevel 

1 

-0.68254 

0.25038 

-2.73 

0.0073 
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Second  level  Regression  Analysis  for  Exercises  1,2,3  52 

12:04  Wednesday,  December  18,  2002 

The  REG  Procedure 
Model:  M0DEL2 
Dependent  Variable:  TP11 
Analysis  of  Variance 


Sum  of 

Mean 

Source 

DF 

Squares 

Square 

F  Value 

Pr  >  F 

Model 

13 

22.63032 

1  .74079 

18.97 

<.0001 

Error 

125 

1 1 .47040 

0.09176 

Corrected  Total 

138 

34.10072 

Root  MSE 

0.30292 

R-Square 

0.6636 

Dependent 

Mean 

0.43165 

Adj  R-Sq 

0.6286 

Coeff  Var 

70.17748 

Parameter  Estimates 

Parameter 

Standard 

Variable 

DF 

Estimate 

Error 

t  Value 

Pr  >  1 1 1 

Intercept 

1 

-0.29709 

0.25403 

-1.17 

0 . 2444 

B0SIntel3 

1 

0.31054 

0.06423 

4.83 

<.0001 

B0SFS2 

1 

0.07143 

0.09898 

0.72 

0.4719 

B0SFS9 

1 

0.15456 

0.09739 

1  .59 

0.1150 

B0SMAS1 

1 

-0.13270 

0.12986 

-1.02 

0.3088 

B0SMAS2 

1 

-0.06189 

0.10381 

-0.60 

0.5522 

B0SCC1 

1 

0.83418 

0.20331 

4.10 

<.0001 

B0SCC2 

1 

0.87388 

0.20922 

4.18 

<.0001 

B0SCC3 

1 

1 .10234 

0.22834 

4.83 

<.0001 

BRIntel 

1 

-0.61101 

0.16319 

-3.74 

0.0003 

Staff Hud 

1 

-0.42061 

0.11265 

-3.73 

0.0003 

PERSXO 

1 

-0.23485 

0.07509 

-3.13 

0.0022 

PERSS3RTO 

1 

0.54579 

0.08475 

6.44 

<.0001 

PERSS3SGM 

1 

-0.39643 

0.08139 

-4.87 

<.0001 
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Second  level  Regression  Analysis  for  Exercises  1,2,3  53 

12:04  Wednesday,  December  18,  2002 

The  REG  Procedure 
Model:  M0DEL3 
Dependent  Variable:  TP12 
Analysis  of  Variance 


Sum  of 

Mean 

Source 

DF 

Squares 

Square 

F  Value 

Pr  >  F 

Model 

17 

6.93516 

0.40795 

12.25 

<.0001 

Error 

121 

4.02886 

0.03330 

Corrected  Total 

138 

10.96403 

Root  MSE 

0.18247 

R-Square 

0.6325 

Dependent 

Mean 

0.08633 

Adj  R-Sq 

0.5809 

Coeff  Var 

211 .36456 

Parameter  Estimates 

Parameter 

Standard 

Variable 

DF 

Estimate 

Error 

t  Value 

Pr  >  1 1 1 

Intercept 

1 

0.93317 

0.11708 

7.97 

<.0001 

B0SFS2 

1 

-0.30422 

0.06577 

-4.63 

<.0001 

B0SFS7 

1 

0.25577 

0.10074 

2.54 

0.0124 

B0SFS9 

1 

-0.20332 

0.04703 

-4.32 

<.0001 

B0SMAS1 

1 

0.05947 

0.06350 

0.94 

0.3509 

B0SMAS4 

1 

-0.06280 

0.09249 

-0.68 

0.4985 

B0SMAS6 

1 

-0.42276 

0.09439 

-4.48 

<.0001 

B0SCC1 

1 

-0.25140 

0.06783 

-3.71 

0.0003 

COMAFATDS 

1 

0.61904 

0.17345 

3.57 

0.0005 

COMASAS 

1 

-0.33603 

0.16757 

-2.01 

0.0472 

COMAMDWS 

1 

0.26328 

0.14526 

1  .81 

0.0724 

MISCBattleTiming 

1 

-1 .29515 

0.15890 

-8.15 

<.0001 

MISCBattleTempo 

1 

0.47793 

0.15783 

3.03 

0.0030 

MISCReconOps 

1 

-0.10782 

0.08478 

-1.27 

0.2059 

PERSXO 

1 

-0.08735 

0.05489 

-1.59 

0.1141 

PERSBC 

1 

0.06549 

0.06936 

0.94 

0.3469 

PERSALO 

1 

-0.13934 

0.04637 

-3.01 

0.0032 

Humidity 

1 

-0.34617 

0.15020 

-2.30 

0.0229 
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Second  level  Regression  Analysis  for  Exercises  1,2,3  54 

12:04  Wednesday,  December  18,  2002 

The  REG  Procedure 
Model:  M0DEL4 
Dependent  Variable:  TP15 
Analysis  of  Variance 


Sum  of 

Mean 

Source 

DF 

Squares 

Square 

F  Value 

Pr  >  F 

Model 

10 

6.60795 

0.66079 

16.34 

<.0001 

Error 

128 

5.17622 

0 . 04044 

Corrected  Total 

138 

1 1 .78417 

Root  MSE 

0.20110 

R-Square 

0.5607 

Dependent 

Mean 

0.09353 

Adj  R-Sq 

0.5264 

Coeff  Var 

215.01712 

Parameter  Estimates 

Parameter 

Standard 

Variable 

DF 

Estimate 

Error 

t  Value 

Pr  >  1 1 1 

Intercept 

1 

-0.14309 

0.06715 

-2.13 

0.0350 

B0SIntel2 

1 

-0.06404 

0.04362 

-1.47 

0.1445 

B0SIntel3 

1 

-0.09813 

0.04450 

-2.21 

0.0292 

B0SIntel4 

1 

0.42428 

0.05810 

7.30 

<.0001 

BOSManl 

1 

-0.08260 

0.05512 

-1.50 

0.1365 

B0SFS2 

1 

0.27076 

0.06451 

4.20 

<.0001 

B0SMAS2 

1 

0.11803 

0.05944 

1  .99 

0.0492 

COMAFATDS 

1 

-0.30406 

0.15664 

-1.94 

0.0544 

MISCBattleTiming 

1 

0.51322 

0.11070 

4.64 

<.0001 

MISCReconOps 

1 

0.01096 

0.08271 

0.13 

0.8948 

PERSXO 

1 

0.15045 

0.04140 

3.63 

0.0004 
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Second  level  Regression  Analysis  for  Exercises  1,2,3 

12:04  Wednesday,  December  1 

The  REG  Procedure 
Model:  MODELS 
Dependent  Variable:  TP21 
Analysis  of  Variance 


Sum  of 

Mean 

Source 

DF 

Squares 

Square 

F  Value 

Pr  >  F 

Model 

6 

14.19017 

2.36503 

27.16 

<.0001 

Error 

132 

1 1 .49329 

0.08707 

Corrected  Total 

138 

25.68345 

Root  MSE 

0.29508 

R-Square 

0.5525 

Dependent 

Mean 

0.24460 

Adj  R-Sq 

0.5322 

Coeff  Var 

120.63437 

Parameter  Estimates 

Parameter 

Standard 

Variable 

DF 

Estimate 

Error 

t  Value 

Pr  >  1 1 1 

Intercept 

1 

0.51864 

0.12822 

4.04 

<.0001 

B0SFS9 

1 

-0.32787 

0.05861 

-5.59 

<.0001 

B0SMAS5 

1 

-0.12771 

0.05582 

-2.29 

0.0237 

MISCBattleTempo 

1 

-0.98918 

0.13539 

-7.31 

<.0001 

MISCInf oOps 

1 

0.46975 

0.13892 

3.38 

0.0009 

Staff Hud 

1 

0.00504 

0.09216 

0.05 

0.9564 

Humidity 

1 

-0.07942 

0.19557 

-0.41 

0.6853 

55 

,  2002 
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Second  level  Regression  Analysis  for  Exercises  1,2,3 

12:04  Wednesday,  December  1 

The  REG  Procedure 
Model:  M0DEL6 
Dependent  Variable:  TP23 
Analysis  of  Variance 


Sum  of 

Mean 

Source 

DF 

Squares 

Square 

F  Value 

Pr  >  F 

Model 

5 

9.11062 

1 .82212 

56.75 

<.0001 

Error 

133 

4.27067 

0.03211 

Corrected  Total 

138 

13.38129 

Root  MSE 

0.17919 

R-Square 

0.6808 

Dependent 

Mean 

0.10791 

Adj  R-Sq 

0.6688 

Coeff  Var 

166.05263 

Parameter  Estimates 

Parameter 

Standard 

Variable 

DF 

Estimate 

Error 

t  Value 

Pr  >  1 1 1 

Intercept 

1 

0.39001 

0.12953 

3.01 

0.0031 

B0SMAS6 

1 

1.01114 

0.06650 

15.20 

<.0001 

BRIntel 

1 

0.31707 

0.08795 

3.60 

0.0004 

BRMAS 

1 

-0.71812 

0.13046 

-5.50 

<.0001 

COMAMDWS 

1 

-0.00067626 

0.10484 

-0.01 

0.9949 

Staff Hud 

1 

0.20796 

0.05590 

3.72 

0.0003 

56 

,  2002 
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Second  level  Regression  Analysis  for  Exercises  1,2,3 

12:04  Wednesday,  December  1 

The  REG  Procedure 
Model:  M0DEL7 
Dependent  Variable:  TS12 
Analysis  of  Variance 


Sum  of 

Mean 

Source 

DF 

Squares 

Square 

F  Value 

Pr  >  F 

Model 

3 

12.02977 

4.00992 

52.40 

<.0001 

Error 

135 

10.32994 

0.07652 

Corrected  Total 

138 

22.35971 

Root  MSE 

0.27662 

R-Square 

0.5380 

Dependent 

Mean 

0.20144 

Adj  R-Sq 

0.5277 

Coeff  Var 

137.32159 

Parameter  Estimates 

Parameter 

Standard 

Variable 

DF 

Estimate 

Error 

t  Value 

Pr  >  1 1 1 

Intercept 

1 

0.04119 

0.03426 

1  .20 

0.2313 

BOSManl 

1 

-0.46001 

0.06273 

-7.33 

<.0001 

B0SFS9 

1 

0.73140 

0.05962 

12.27 

<.0001 

B0SMAS1 

1 

-0.28259 

0.07846 

-3.60 

0.0004 

57 

,  2002 
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Second  level  Regression  Analysis  for  Exercises  1,2,3  58 

12:04  Wednesday,  December  18,  2002 

The  REG  Procedure 
Model:  M0DEL8 
Dependent  Variable:  TS18 
Analysis  of  Variance 


Sum  of 

Mean 

Source 

DF 

Squares 

Square 

F  Value 

Pr  >  F 

Model 

8 

3.19045 

0.39881 

5.52 

<.0001 

Error 

130 

9.39948 

0.07230 

Corrected  Total 

138 

12.58993 

Root  MSE 

0.26889 

R-Square 

0.2534 

Dependent 

Mean 

0.10072 

Adj  R-Sq 

0.2075 

Coeff  Var 

266.97277 

Parameter  Estimates 

Parameter 

Standard 

Variable 

DF 

Estimate 

Error 

t  Value 

Pr  >  1 1 1 

Intercept 

1 

-0.11096 

0.10355 

-1.07 

0.2859 

B0SFS2 

1 

0.24509 

0.08501 

2.88 

0.0046 

B0SMAS6 

1 

-0.35365 

0.12312 

-2.87 

0.0048 

B0SCC1 

1 

0.25674 

0.08340 

3.08 

0.0025 

COMAFATDS 

1 

-0.28635 

0.19078 

-1.50 

0.1358 

MISCBattleTiming 

1 

0.39689 

0.20214 

1  .96 

0.0517 

MISCReconOps 

1 

0.05850 

0.11214 

0.52 

0.6028 

MISCInf oOps 

1 

-0.13715 

0.12498 

-1 .10 

0.2745 

PERSXO 

1 

0.28131 

0.05552 

5.07 

<.0001 
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Second  level  Regression  Analysis  for  Exercises  1,2,3 

12:04  Wednesday,  December  1 

The  REG  Procedure 
Model:  M0DEL9 
Dependent  Variable:  TS19 
Analysis  of  Variance 


Sum  of 

Mean 

Source 

DF 

Squares 

Square 

F  Value 

Pr  >  F 

Model 

2 

7.27184 

3.63592 

133.93 

<.0001 

Error 

136 

3.69219 

0.02715 

Corrected  Total 

138 

10.96403 

Root  MSE 

0.16477 

R-Square 

0.6632 

Dependent 

Mean 

0.08633 

Adj  R-Sq 

0.6583 

Coeff  Var 

190.85602 

Parameter  Estimates 

Parameter 

Standard 

Variable 

DF 

Estimate 

Error 

t  Value 

Pr  >  1 1 1 

Intercept 

1 

0.09891 

0.02985 

3.31 

0.0012 

B0SMAS6 

1 

0.96695 

0.06001 

16.11 

<.0001 

PERSS2 

1 

-0.08781 

0.03358 

-2.62 

0.0099 

59 

,  2002 
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Second  level  Regression  Analysis  for  Exercises  1,2,3  60 

12:04  Wednesday,  December  18,  2002 

The  REG  Procedure 
Model:  M0DEL10 
Dependent  Variable:  TS23 
Analysis  of  Variance 


Sum  of 

Mean 

Source 

DF 

Squares 

Square 

F  Value 

Pr  >  F 

Model 

10 

8.38067 

0.83807 

14.72 

<.0001 

Error 

128 

7.28840 

0.05694 

Corrected  Total 

138 

15.66906 

Root  MSE 

0.23862 

R-Square 

0.5349 

Dependent 

Mean 

0.12950 

Adj  R-Sq 

0.4985 

Coeff  Var 

184.26947 

Parameter  Estimates 

Parameter 

Standard 

Variable 

DF 

Estimate 

Error 

t  Value 

Pr  >  1 1 1 

Intercept 

1 

-0.50056 

0.15867 

-3.15 

0.0020 

B0SIntel2 

1 

0.12448 

0.05538 

2.25 

0.0263 

B0SIntel3 

1 

0.20154 

0.05547 

3.63 

0.0004 

B0SMan2 

1 

-0.23144 

0.08071 

-2.87 

0.0048 

B0SFS9 

1 

-0.13123 

0.05672 

-2.31 

0.0223 

B0SMAS4 

1 

0.58920 

0.11780 

5.00 

<.0001 

B0SCC2 

1 

0.18261 

0.05731 

3.19 

0.0018 

MISCBattleTiming 

1 

-0.52596 

0.15863 

-3.32 

0.0012 

MISCInf oOps 

1 

0.26852 

0.11073 

2.43 

0.0167 

PERSS3RT0 

1 

0.35949 

0.06329 

5.68 

<.0001 

Humidity 

1 

0.58127 

0.18105 

3.21 

0.0017 
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Second  level  Regression  Analysis  for  Exercises  1,2,3  61 

12:04  Wednesday,  December  18,  2002 

The  REG  Procedure 
Model:  MODEL11 
Dependent  Variable:  TA3 
Analysis  of  Variance 


Sum  of 

Mean 

Source 

DF 

Squares 

Square 

F  Value 

Pr  >  F 

Model 

10 

7.63123 

0.76312 

29.31 

<.0001 

Error 

128 

3.33280 

0.02604 

Corrected  Total 

138 

10.96403 

Root  MSE 

0.16136 

R-Square 

0.6960 

Dependent 

Mean 

0.08633 

Adj  R-Sq 

0.6723 

Coeff  Var 

186.91015 

Parameter  Estimates 

Parameter 

Standard 

Variable 

DF 

Estimate 

Error 

t  Value 

Pr  >  | t | 

Intercept 

1 

0.08980 

0 . 04943 

1  .82 

0.0716 

BOSManl 

1 

0.40832 

0.04661 

8.76 

<.0001 

B0SFS2 

1 

0.23808 

0.05374 

4.43 

<.0001 

B0SFS9 

1 

-0.14227 

0.04457 

-3.19 

0.0018 

B0SMAS1 

1 

-0.39912 

0.05216 

-7.65 

<.0001 

B0SCC1 

1 

-0.15818 

0.04280 

-3.70 

0.0003 

COMAFATDS 

1 

-0.73340 

0.12275 

-5.97 

<.0001 

COMMCS 

1 

0.49004 

0.10305 

4.76 

<.0001 

Staff Hud 

1 

-0.28177 

0.05568 

-5.06 

<.0001 

PERSS3RTO 

1 

0.11013 

0.03845 

2.86 

0.0049 

PERSALO 

1 

-0.09726 

0.03674 

-2.65 

0.0091 
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Second  level  Regression  Analysis  for  Exercises  1,2,3  62 

12:04  Wednesday,  December  18,  2002 

The  REG  Procedure 
Model:  M0DEL12 
Dependent  Variable:  TA21 
Analysis  of  Variance 


Sum  of 

Mean 

Source 

DF 

Squares 

Square 

F  Value 

Pr  >  F 

Model 

14 

16.53220 

1 .18087 

73.74 

<.0001 

Error 

124 

1 .98579 

0.01601 

Corrected  Total 

138 

18.51799 

Root  MSE 

0.12655 

R-Square 

0.8928 

Dependent 

Mean 

0.15827 

Adj  R-Sq 

0.8807 

Coeff  Var 

79.95537 

Parameter  Estimates 

Parameter 

Standard 

Variable 

DF 

Estimate 

Error 

t  Value 

Pr  >  1 1 1 

Intercept 

1 

-0.02767 

0.07718 

-0.36 

0.7206 

BOSIntell 

1 

-0.28954 

0.04924 

-5.88 

<.0001 

BOSManl 

1 

-0.45515 

0.04625 

-9.84 

<.0001 

B0SFS9 

1 

0.54835 

0.04903 

11.18 

<.0001 

B0SMAS1 

1 

-0.09739 

0.05770 

-1.69 

0.0939 

B0SMAS7 

1 

-0.14943 

0.05374 

-2.78 

0.0063 

B0SCC1 

1 

0.11514 

0.05359 

2.15 

0.0336 

B0SCC3 

1 

0.09074 

0.05007 

1  .81 

0.0724 

COMAFATDS 

1 

-0.15557 

0.08974 

-1.73 

0.0855 

MISCBattleTiming 

1 

0.44087 

0.14014 

3.15 

0.0021 

MISCBattleTempo 

1 

-0.46725 

0.10193 

-4.58 

<.0001 

MISCTOCActivity 

1 

-0.11781 

0.05948 

-1.98 

0.0498 

Staff Hud 

1 

-0.13192 

0.04489 

-2.94 

0.0039 

PERSS3RTO 

1 

0.33150 

0.04868 

6.81 

<.0001 

PERSALO 

1 

-0.08447 

0.02929 

-2.88 

0.0046 
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Second  level  Regression  Analysis  for  Exercises  1,2,3  63 

12:04  Wednesday,  December  18,  2002 

The  REG  Procedure 
Model:  MODEL13 
Dependent  Variable:  TA23 
Analysis  of  Variance 


Sum  of 

Mean 

Source 

DF 

Squares 

Square 

F  Value 

Pr  >  F 

Model 

21 

20.12878 

0.95851 

68.94 

<.0001 

Error 

117 

1 .62662 

0.01390 

Corrected  Total 

138 

21 .75540 

Root  MSE 

0.11791 

R-Square 

0.9252 

Dependent 

Mean 

0.19424 

Adj  R-Sq 

0.9118 

Coeff  Var 

60.70175 

Parameter  Estimates 

Parameter 

Standard 

Variable 

DF 

Estimate 

Error 

t  Value 

Pr  >  1 1 1 

Intercept 

1 

-0.46695 

0.24447 

-1.91 

0.0586 

BOSIntell 

1 

0.67298 

0.08156 

8.25 

<.0001 

B0SIntel2 

1 

0.63978 

0.06405 

9.99 

<.0001 

B0SIntel3 

1 

0.68694 

0.06932 

9.91 

<.0001 

B0SIntel4 

1 

0.86051 

0.08437 

10.20 

<.0001 

B0SFS9 

1 

0.21460 

0.04594 

4.67 

<.0001 

B0SMAS1 

1 

-0.09621 

0.05589 

-1.72 

0.0878 

B0SMAS6 

1 

0.71439 

0.11851 

6.03 

<.0001 

B0SADA1 

1 

-1 .14523 

0.22691 

-5.05 

<.0001 

B0SADA2 

1 

-1.18712 

0.18938 

-6.27 

<.0001 

B0SCC1 

1 

0.41226 

0.10360 

3.98 

0.0001 

B0SCC2 

1 

0.21302 

0.10626 

2.00 

0.0473 

B0SCC3 

1 

0.19935 

0.11909 

1  .67 

0.0968 

BRIntel 

1 

-0.71353 

0.08259 

-8.64 

<.0001 

BRMAS 

1 

0.08540 

0.15544 

0.55 

0.5838 

MISCBattleTiming 

1 

0.58805 

0.15234 

3.86 

0.0002 

MOPPLevel 

1 

2.37320 

0.38672 

6.14 

<.0001 

Staff Hud 

1 

0.18661 

0.04709 

3.96 

0.0001 

PERSBC 

1 

0.06271 

0.0471 1 

1  .33 

0.1857 

PERSS3 

1 

0.03461 

0.03045 

1.14 

0.2581 

PERSS3RTO 

1 

-0.19467 

0.05600 

-3.48 

0.0007 

Humidity 

1 

-0.22699 

0.10483 

-2.17 

0.0324 
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Second  level  Regression  Analysis  for  Exercises  1,2,3  64 

12:04  Wednesday,  December  18,  2002 

The  REG  Procedure 
Model:  M0DEL14 
Dependent  Variable:  Missionl 
Analysis  of  Variance 


Sum  of 

Mean 

Source 

DF 

Squares 

Square 

F  Value 

Pr  >  F 

Model 

10 

9.25870 

0.92587 

19.88 

<.0001 

Error 

128 

5.96202 

0.04658 

Corrected  Total 

138 

15.22072 

Root  MSE 

0.21582 

R-Square 

0.6083 

Dependent 

Mean 

0.16835 

Adj  R-Sq 

0.5777 

Coeff  Var 

128.20076 

Parameter  Estimates 

Parameter 

Standard 

Variable 

DF 

Estimate 

Error 

t  Value 

Pr  >  1 1 1 

Intercept 

1 

0.45405 

0.09039 

5.02 

<.0001 

B0SIntel2 

1 

-0.07652 

0.05406 

-1.42 

0.1594 

B0SIntel3 

1 

0.02792 

0.05511 

0.51 

0.6133 

B0SMAS1 

1 

-0.36447 

0.07960 

-4.58 

<.0001 

B0SMAS2 

1 

0.49815 

0.07485 

6.66 

<.0001 

B0SMAS4 

1 

0.55403 

0.11781 

4.70 

<.0001 

B0SADA1 

1 

-0.22545 

0.07917 

-2.85 

0.0051 

COMMCS 

1 

0.49437 

0.13737 

3.60 

0.0005 

MISCBattleTiming 

1 

-1 .00380 

0.14116 

-7.11 

<.0001 

PERSS3RTO 

1 

-0.09414 

0.05131 

-1.83 

0.0689 

PERSS3SGM 

1 

-0.11605 

0.05786 

-2.01 

0.0470 
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Second  level  Regression  Analysis  for  Exercises  1,2,3 

12:04  Wednesday,  December  1 

The  REG  Procedure 
Model:  M0DEL15 
Dependent  Variable:  Mission2 
Analysis  of  Variance 


Sum  of 

Mean 

Source 

DF 

Squares 

Square 

F  Value 

Pr  >  F 

Model 

9 

3.51003 

0.39000 

19.92 

<.0001 

Error 

129 

2.52522 

0.01958 

Corrected  Total 

138 

6.03525 

Root  MSE 

0.13991 

R-Square 

0.5816 

Dependent 

Mean 

0.22158 

Adj  R-Sq 

0.5524 

Coeff  Var 

63.14212 

Parameter  Estimates 

Parameter 

Standard 

Variable 

DF 

Estimate 

Error 

t  Value 

Pr  >  1 1 1 

Intercept 

1 

0.16123 

0.08040 

2.01 

0.0470 

B0SFS2 

1 

-0.09902 

0.04414 

-2.24 

0.0266 

B0SMAS1 

1 

0.08027 

0.03961 

2.03 

0.0448 

B0SCC2 

1 

0.22204 

0.03505 

6.34 

<■0001 

B0SCC3 

1 

0.20824 

0.03951 

5.27 

<.0001 

BRIntel 

1 

0.04372 

0.07499 

0.58 

0.5609 

COMAFATDS 

1 

0.10406 

0.10840 

0.96 

0.3389 

MISCReconOps 

1 

-0.14001 

0.05782 

-2.42 

0.0168 

Staff Hud 

1 

0.00998 

0.04430 

0.23 

0.8221 

PERSS3NC0 

1 

-0.22587 

0.03099 

-7.29 

<.0001 
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Second  level  Regression  Analysis  for  Exercises  1,2,3  66 

12:04  Wednesday,  December  18,  2002 

The  REG  Procedure 
Model:  M0DEL16 
Dependent  Variable:  Mission3 
Analysis  of  Variance 


Sum  of 

Mean 

Source 

DF 

Squares 

Square 

F  Value 

Pr  >  F 

Model 

12 

3.17299 

0.26442 

9.00 

<.0001 

Error 

126 

3.70219 

0.02938 

Corrected  Total 

138 

6.87518 

Root  MSE 

0.17141 

R-Square 

0.4615 

Dependent 

Mean 

0.08417 

Adj  R-Sq 

0.4102 

Coeff  Var 

203.64463 

Parameter  Estimates 

Parameter 

Standard 

Variable 

DF 

Estimate 

Error 

t  Value 

Pr  >  1 1 1 

Intercept 

1 

-0.65866 

0.22134 

-2.98 

0.0035 

B0SIntel4 

1 

0.28178 

0.05314 

5.30 

<.0001 

B0SFS7 

1 

0.03883 

0.08716 

0.45 

0.6567 

B0SMAS4 

1 

-0.20905 

0.09304 

-2.25 

0.0264 

BRMAS 

1 

0.39083 

0.21102 

1  .85 

0.0664 

COMMCS 

1 

-0.28729 

0.11644 

-2.47 

0.0150 

INTELTUAV 

1 

-0.03119 

0.07467 

-0.42 

0.6769 

MISCBattleTiming 

1 

0.42767 

0.15229 

2.81 

0.0058 

MISCBattleTempo 

1 

-0.03118 

0.16911 

-0.18 

0.8540 

MOPPLevel 

1 

0.55691 

0.17701 

3.15 

0.0021 

PERSCMDR 

1 

0.02565 

0.04637 

0.55 

0.5812 

PERSXO 

1 

0.08611 

0.04634 

1 .86 

0.0655 

Humidity 

1 

0.25756 

0.13464 

1  .91 

0.0580 
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Second  level  Regression  Analysis  for  Exercises  1,2,3  67 

12:04  Wednesday,  December  18,  2002 

The  REG  Procedure 
Model:  M0DEL17 

Dependent  Variable:  Mission5 
Analysis  of  Variance 


Sum  of 

Mean 

Source 

DF 

Squares 

Square 

F  Value 

Pr  >  F 

Model 

1 1 

4.04591 

0.36781 

19.77 

<.0001 

Error 

127 

2.36315 

0.01861 

Corrected  Total 

138 

6.40906 

Root  MSE 

0.13641 

R-Square 

0.6313 

Dependent 

Mean 

0.07050 

Adj  R-Sq 

0.5993 

Coeff  Var 

193.47854 

Parameter  Estimates 

Parameter 

Standard 

Variable 

DF 

Estimate 

Error 

t  Value 

Pr  >  1 1 1 

Intercept 

1 

-0.70717 

0.06980 

-10.13 

<.0001 

B0SIntel3 

1 

-0.03968 

0.03086 

-1.29 

0.2009 

B0SIntel4 

1 

0.10022 

0.04249 

2.36 

0.0199 

BOSManl 

1 

0.38117 

0.06082 

6.27 

<.0001 

B0SFS9 

1 

-0.03477 

0.03193 

-1.09 

0.2782 

B0SADA2 

1 

0.40088 

0.04714 

8.50 

<.0001 

COMAFATDS 

1 

-0.17253 

0.10141 

-1.70 

0.0913 

COMASAS 

1 

0.16611 

0.11209 

1  .48 

0.1408 

MISCBattleTiming 

1 

1  .28222 

0.11669 

10.99 

<.0001 

MOPPLevel 

1 

-0.73388 

0.13344 

-5.50 

<.0001 

PERSS3NC0 

1 

0.12560 

0.03324 

3.78 

0.0002 

PERSALO 

1 

0.06181 

0.03226 

1  .92 

0.0576 
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Appendix  L  -  Third  Linear  Regression  Run  To  Select  IV  For  Inclusion  In  DV  Models 


SAS  Code  (Pick  those  IV  that  had  a  significant p  value  (a  =  .05)  from  the  Second  run): 

options  pageno=l  f ormdlim="-" ; 

TITLE  "Training  Data  123  Regression  Analysis'; 

DATA  Tngdatal23; 

INPUT  Dayl  Day2  Day3  Day4  Day5  Day6  Day7  Day8  Day9  DaylO  Dayll  Dayl2  Dayl3  Dayl4  Dayl5  Dayl6 
Dayl7  Dayl8  Dayl9  Day20  Hourl  Hour2  Hour3  Hour4  Hour5  Hour6  Hour7  Hour8  Hour9  HourlO  Hourll 
Hourl2  Hourl3  Hourl4  Hourl5  Hourl6  Hourl7  Hourl8  Hourl9  Hour20  Hour21  Hour22  Hour23  Hour24 
TOCAn  TOCDg  TOCBn  TOCBde  ExTypl  ExTyp2  ExTyp3  ExTyp4  ExNol  ExNo2  ExNo3  ExNo4  ExNo5  ExNo6  ExNo7 
ExNo8  ExNo9  ExNolO  ExNoll  ExNol2  ExNol3  ExNol4  ExNol5  ExNol6  ExNol7  ExNol8  ExNol9  ExNo20  SimNo 
SimJANS  SimJAST  SimCBS  SimCCTT  SimSTRM  BOSIntell  BOSIntel2  BOSIntel3  BOSIntel4  BOSManl  BOSMan2 
BOSFS1  BOSFS2  BOSFS3  BOSFS4  BOSFS5  BOSFS6  BOSFS7  BOSFS8  BOSFS9  BOSMAS1  BOSMAS2  BOSMAS3  BOSMAS4 
BOSMAS5  BOSMAS6  BOSMAS7  BOSADA1  BOSADA2  BOSCC1  BOSCC2  BOSCC3  BRIntel  BRMan  BRFS  BRMAS  BRADA 
BRCSS  BRC2  COMAFATDS  COMAMPS  COMASAS  COMCTIS  COMCSSCS  COMFAADC2  COMFBCB2  COMIMETS  COMMCS  COMAMDWS 
COMJSTARS  COMEPLARS  INTELTUAV  INTELJSTARS  INTELGSR  MISCBattleTiming  MISCBattleTempo 
MISCReconOps  MISCInfoOps  MISCTactics  MISCTOCActivity  MOPPLevel  StaffHud  PERSCMDR  PERSXO  PERSBC 
PERSS3  PERSS3NCO  PERSS3RTO  PERSS2  PERSFSO  PERSENGR  PERSALO  PERSS3SGM  SKYCLR  SKYOC  SKYRAIN 
WINDNONE  WINDLOW  WINDMOD  WINDHIGH  LightBG  LightFG  Noise  TEMPDB  TEMPWB  Humidity  TP1  TP2  TP3  TP4 
TP 5  TP 6  TP 7  TP 8  TP 9  TP10  TP11  TP12  TP13  TP14  TP15  TP16  TP17  TP18  TP19  TP20  TP21  TP22  TP23  TS1 
TS2  TS3  TS4  TS5  TS6  TS7  TS8  TS9  TS10  TS11  TS12  TS13  TS14  TS15  TS16  TS17  TS18  TS19  TS20  TS21 
TS22  TS23  TA1  TA2  TA3  TA4  TA5  TA6  TA7  TA8  TA9  TA10  TA11  TA12  TA13  TA14  TA15  TA16  TA17  TA18 
TA19  TA20  TA21  TA22  TA23  Missionl  Mission2  Mission3  Mission4  Mission5  Mission6 
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*  Regression  third  level  run; 

*  Partial  Tng  Regression  Analysis; 

*  Make  regression  run  with  input/independent  variables  remaining  that  are  significant  after  full 
regression  run 

*  to  try  to  maximize  the  adjusted  r**2  showing  the  amount  of  the  regression  accounted  for; 
options  pageno=l  f ormdlim="-" ; 

proc  reg  data=Tngdatal23; 

TITLE  "Third  level  Regression  Analysis  for  Exercises  1,2,3"; 

*  model  TP2=BOSFSl  BOSFS2  BOSFS3  BOSFS7  BOSFS9; 
model  TP10=BOSIntel4  BOSMAS2  BOSCC2  BOSCC3  MOPPLevel; 

model  TP1 l=BOSIntel3  BOSCC1  BOSCC2  BOSCC3  BRIntel  StaffHud  PERSXO  PERSS3RTO  PERSS3SGM; 
model  TP12=BOSFS2  BOSFS7  BOSFS9  BOSMAS6  BOSCC1  COMAFATDS  COMASAS  MISCBattleTiming 
MISCBattleTempo  PERSALO  Humidity; 

model  TP15=BOSIntel3  BOSIntel4  BOSFS2  BOSMAS2  COMAFATDS  MISCBattleTiming  PERSXO  ; 
model  TP21=BOSFS9  BOSMAS5  MISCBattleTempo  MISCInfoOps  ; 
model  TP2 3=BOSMAS 6  BRIntel  BRMAS  StaffHud; 
model  TS12=BOSManl  BOSFS9  BOSMAS1; 

model  TS18=BOSFS2  BOSMAS6  BOSCC1  MISCBattleTiming  PERSXO  ; 
model  TS19=BOSMAS6  PERSS2 ; 

model  TS23=BOSIntel2  BOSIntel3  BOSMan2  BOSFS9  BOSMAS4  BOSCC2  MISCBattleTiming  MISCInfoOps 
PERSS3RTO  Humidity; 

model  TA3=BOSManl  BOSFS2  BOSFS9  BOSMAS1  BOSCC1  COMAFATDS  COMMCS  StaffHud  PERSS3RTO  PERSALO; 
model  TA21=BOSIntell  BOSManl  BOSFS9  BOSMAS7  BOSCC1  COMAFATDS  MISCBattleTiming  MISCBattleTempo 
MISCTOCActivity  StaffHud  PERSS3RTO  PERSALO; 

model  TA23=BOSIntell  BOSIntel2  BOSIntel3  BOSIntel4  BOSFS9  BOSMAS6  BOSADA1  BOSADA2  BOSCC1  BOSCC2 
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BRIntel  MISCBattleTiming  MOPPLevel  StaffHud  PERSS3RTO  Humidity; 
model  Missionl=BOSMASl  BOSMAS2  BOSMAS4  BOSADA1  COMMCS  MISCBattleTiming  PERSS3SGM 
model  Mission2=BOSFS2  BOSMAS1  BOSCC2  BOSCC3  MISCReconOps  PERSS3NCO  ; 
model  Mission3=BOSIntel4  BOSMAS4  COMMCS  MISCBattleTiming  MOPPLevel  ; 
model  Mission5=BOSIntel4  BOSManl  BOSADA2  MISCBattleTiming  MOPPLevel  PERSS3NCO; 
run; 
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SAS  Output: 


Third  level  Regression  Analysis  for  Exercises  1,2,3 

12:04  Wednesday, 


The  REG  Procedure 


December  1 


Model:  M0DEL1 


Dependent  Variable:  TP10 


Analysis  of  Variance 


Sum  of 

Mean 

Source 

DF 

Squares 

Square 

F  Value 

Pr  >  F 

Model 

5 

7.51355 

1 .50271 

34.06 

<.0001 

Error 

133 

5.86775 

0.04412 

Corrected  Total 

138 

13.38129 

Root  MSE 

0.21004 

R-Square 

0.5615 

Dependent 

Mean 

0.10791 

Adj  R-Sq 

0.5450 

Coeff  Var 

194.64058 

Parameter  Estimates 

Parameter 

Standard 

Variable 

DF 

Estimate 

Error 

t  Value 

Pr  >  1 1 1 

Intercept 

1 

0.06734 

0.05007 

1  .35 

0.1809 

B0SIntel4 

1 

0.36773 

0.04806 

7.65 

<.0001 

B0SMAS2 

1 

0.30461 

0.05644 

5.40 

<.0001 

B0SCC2 

1 

-0.16734 

0.05570 

-3.00 

0.0032 

B0SCC3 

1 

0.04340 

0.05602 

0.77 

0.4399 

MOPPLevel 

1 

-0.43653 

0.12104 

-3.61 

0.0004 

Third  level  Regression  Analysis  for  Exercises  1,2,3 

12:04  Wednesday,  December  1 

The  REG  Procedure 
Model:  M0DEL2 
Dependent  Variable:  TP11 
Analysis  of  Variance 


Sum  of 

Mean 

Source 

DF 

Squares 

Square 

F  Value 

Pr  >  F 

Model 

9 

22.29580 

2.47731 

27.07 

<.0001 

Error 

129 

1 1 .80492 

0.09151 

Corrected  Total 

138 

34.10072 

Root  MSE 

0.30251 

R-Square 

0.6538 

Dependent 

Mean 

0.43165 

Adj  R-Sq 

0.6297 

Coeff  Var 

70.08099 

Parameter  Estimates 


Variable  DF 
Intercept  1 
B0SIntel3  1 
B0SCC1  1 
B0SCC2  1 
B0SCC3  1 
BRIntel  1 
StaffHud  1 
PERSXO  1 
PERSS3RTO  1 
PERSS3SGM  1 


Parameter 

Standard 

Estimate 

Error 

-0.26097 

0.24789 

0.27295 

0.05763 

0.85678 

0.18988 

0.85483 

0 . 1 9643 

1 .15989 

0.20510 

-0.61238 

0.15375 

-0.39140 

0.09395 

-0.27882 

0.06923 

0.58359 

0.07289 

-0.38158 

0.07344 

t  Value 

Pr  >  1 1 1 

IT) 

O 

0.2944 

4.74 

<.0001 

4.51 

<.0001 

4.35 

<.0001 

5.66 

<.0001 

-3.98 

0.0001 

-4.17 

<.0001 

-4.03 

<.0001 

8.01 

<.0001 

-5.20 

<.0001 

1 

,  2002 


2 

,  2002 


425 


Third  level  Regression  Analysis  for  Exercises  1,2,3  3 

12:04  Wednesday,  December  18,  2002 

The  REG  Procedure 
Model:  MODEL3 
Dependent  Variable:  TP12 
Analysis  of  Variance 


Sum  of 

Mean 

Source 

DF 

Squares 

Square 

F  Value 

Pr  >  F 

Model 

1 1 

6.59201 

0.59927 

17.41 

<.0001 

Error 

127 

4.37202 

0.03443 

Corrected  Total 

138 

10.96403 

Root  MSE 

0.18554 

R-Square 

0.6012 

Dependent 

Mean 

0.08633 

Adj  R-Sq 

0.5667 

Coeff  Var 

214.91796 

Parameter  Estimates 

Parameter 

Standard 

Variable 

DF 

Estimate 

Error 

t  Value 

Pr  >  | t | 

Intercept 

1 

1 .00078 

0.09816 

10.20 

<.0001 

B0SFS2 

1 

-0.25704 

0.06338 

-4.06 

<.0001 

B0SFS7 

1 

0.34609 

0.09585 

3.61 

0.0004 

B0SFS9 

1 

-0.17123 

0.04266 

-4.01 

0.0001 

B0SMAS6 

1 

-0.39444 

0.09020 

-4.37 

<.0001 

B0SCC1 

1 

-0.28068 

0.06474 

-4.34 

<.0001 

COMAFATDS 

1 

0.47345 

0.16034 

2.95 

0.0038 

COMASAS 

1 

-0.09658 

0.13655 

-0.71 

0.4807 

MISCBattleTiming 

1 

-1 .15387 

0.14497 

-7.96 

<.0001 

MISCBattleTempo 

1 

0.18928 

0.10964 

1  .73 

0.0867 

PERSALO 

1 

-0.15613 

0.04215 

-3.70 

0.0003 

Humidity 

1 

-0.46752 

0.13826 

-3.38 

0.0010 

Third  level  Regression  Analysis  for  Exercises  1,2,3  4 

12:04  Wednesday,  December  18,  2002 

The  REG  Procedure 
Model:  M0DEL4 
Dependent  Variable:  TP15 
Analysis  of  Variance 


Sum  of 

Mean 

Source 

DF 

Squares 

Square 

F  Value 

Pr  >  F 

Model 

7 

6.46417 

0.92345 

22.74 

<.0001 

Error 

131 

5.32001 

0.04061 

Corrected  Total 

138 

1 1 .78417 

Root  MSE 

0.20152 

R-Square 

0.5485 

Dependent 

Mean 

0.09353 

Adj  R-Sq 

0.5244 

Coeff  Var 

215.47250 

Parameter  Estimates 

Parameter 

Standard 

Variable 

DF 

Estimate 

Error 

t  Value 

Pr  >  1 1 1 

Intercept 

1 

-0.21474 

0.05254 

-4.09 

<.0001 

B0SIntel3 

1 

-0.07527 

0.04227 

-1.78 

0.0773 

B0SIntel4 

1 

0.38302 

0.05236 

7.31 

<.0001 

B0SFS2 

1 

0.26797 

0.06303 

4.25 

<.0001 

B0SMAS2 

1 

0.07394 

0.04446 

1  .66 

0.0987 

COMAFATDS 

1 

-0.26475 

0.14546 

-1.82 

0.0710 

MISCBattleTiming 

1 

0.52821 

0.10705 

4.93 

<.0001 

PERSXO 

1 

0.14446 

0.04134 

3.49 

0.0006 
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Third  level  Regression  Analysis  for  Exercises  1,2,3 

12:04  Wednesday,  December  1 

The  REG  Procedure 
Model:  MODELS 
Dependent  Variable:  TP21 
Analysis  of  Variance 


Sum  of 

Mean 

Source 

DF 

Squares 

Square 

F  Value 

Pr  >  F 

Model 

4 

14.17579 

3.54395 

41 .27 

<.0001 

Error 

134 

11 .50766 

0.08588 

Corrected  Total 

138 

25.68345 

Root  MSE 

0.29305 

R-Square 

0.5519 

Dependent 

Mean 

0.24460 

Adj  R-Sq 

0.5386 

Coeff  Var 

119.80558 

Parameter  Estimates 

Parameter 

Standard 

Variable 

DF 

Estimate 

Error 

t  Value 

Pr  >  1 1 1 

Intercept 

1 

0.47554 

0.07136 

6.66 

<.0001 

B0SFS9 

1 

-0.32588 

0.05679 

-5.74 

<.0001 

B0SMAS5 

1 

-0.12732 

0.05531 

-2.30 

0.0229 

MISCBattleTempo 

1 

-0.97917 

0.13160 

-7.44 

<.0001 

MISCInf oOps 

1 

0.45966 

0.13416 

3.43 

0.0008 

Third  level  Regression  Analysis  for  Exercises  1,2,3 

12:04  Wednesday,  December  1 

The  REG  Procedure 
Model:  M0DEL6 
Dependent  Variable:  TP23 
Analysis  of  Variance 


Sum  of 

Mean 

Source 

DF 

Squares 

Square 

F  Value 

Pr  >  F 

Model 

4 

9.11062 

2.27766 

71  .47 

<.0001 

Error 

134 

4.27067 

0.03187 

Corrected  Total 

138 

13.38129 

Root  MSE 

0.17852 

R-Square 

0.6808 

Dependent 

Mean 

0.10791 

Adj  R-Sq 

0.6713 

Coeff  Var 

165.43190 

Parameter  Estimates 

Parameter 

Standard 

Variable 

DF 

Estimate 

Error 

t  Value 

Pr  >  1 1 1 

Intercept 

1 

0.39034 

0.11841 

3.30 

0.0013 

B0SMAS6 

1 

1  .01 120 

0.06566 

15.40 

<.0001 

BRIntel 

1 

0.31699 

0.08691 

3.65 

0.0004 

BRMAS 

1 

-0.71853 

0.11348 

-6.33 

<.0001 

Staff Hud 

1 

0.20794 

0.05561 

3.74 

0.0003 

5 

,  2002 


6 

,  2002 
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Third  level  Regression  Analysis  for  Exercises  1,2,3 

12:04  Wednesday,  December  1 

The  REG  Procedure 
Model:  M0DEL7 
Dependent  Variable:  TS12 
Analysis  of  Variance 


Sum  of 

Mean 

Source 

DF 

Squares 

Square 

F  Value 

Pr  >  F 

Model 

3 

12.02977 

4.00992 

52.40 

<.0001 

Error 

135 

10.32994 

0.07652 

Corrected  Total 

138 

22.35971 

Root  MSE 

0.27662 

R-Square 

0.5380 

Dependent 

Mean 

0.20144 

Adj  R-Sq 

0.5277 

Coeff  Var 

137.32159 

Parameter  Estimates 

Parameter 

Standard 

Variable 

DF 

Estimate 

Error 

t  Value 

Pr  >  1 1 1 

Intercept 

1 

0.04119 

0.03426 

1  .20 

0.2313 

BOSManl 

1 

-0.46001 

0.06273 

-7.33 

<.0001 

B0SFS9 

1 

0.73140 

0.05962 

12.27 

<.0001 

B0SMAS1 

1 

-0.28259 

0.07846 

-3.60 

0.0004 

Third  level  Regression  Analysis  for  Exercises  1,2,3 

12:04  Wednesday,  December  1 

The  REG  Procedure 
Model:  M0DEL8 
Dependent  Variable:  TS18 
Analysis  of  Variance 


Sum  of 

Mean 

Source 

DF 

Squares 

Square 

F  Value 

Pr  >  F 

Model 

5 

2.93386 

0.58677 

8.08 

<.0001 

Error 

133 

9.65607 

0.07260 

Corrected  Total 

138 

12.58993 

Root  MSE 

0.26945 

R-Square 

0.2330 

Dependent 

Mean 

0.10072 

Adj  R-Sq 

0.2042 

Coeff  Var 

267.52295 

Parameter  Estimates 

Parameter 

Standard 

Variable 

DF 

Estimate 

Error 

t  Value 

Pr  >  1 1 1 

Intercept 

1 

-0.14761 

0.09511 

-1.55 

0.1230 

B0SFS2 

1 

0.15261 

0.06459 

2.36 

0.0196 

B0SMAS6 

1 

-0.31555 

0.11629 

-2.71 

0.0075 

B0SCC1 

1 

0.22247 

0.08148 

2.73 

0.0072 

MISCBattleTiming 

1 

0.26665 

0.18858 

1  .41 

0.1597 

PERSXO 

1 

0.24460 

0.05043 

4.85 

<.0001 

7 

,  2002 


8 

,  2002 
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Third  level  Regression  Analysis  for  Exercises  1,2,3 

12:04  Wednesday,  December  1 

The  REG  Procedure 
Model:  M0DEL9 
Dependent  Variable:  TS19 
Analysis  of  Variance 


Sum  of 

Mean 

Source 

DF 

Squares 

Square 

F  Value 

Pr  >  F 

Model 

2 

7.27184 

3.63592 

133.93 

<.0001 

Error 

136 

3.69219 

0.02715 

Corrected  Total 

138 

10.96403 

Root  MSE 

0.16477 

R-Square 

0.6632 

Dependent 

Mean 

0.08633 

Adj  R-Sq 

0.6583 

Coeff  Var 

190.85602 

Parameter  Estimates 

Parameter 

Standard 

Variable 

DF 

Estimate 

Error 

t  Value 

Pr  >  1 1 1 

Intercept 

1 

0.09891 

0.02985 

3.31 

0.0012 

B0SMAS6 

1 

0.96695 

0.06001 

16.11 

<.0001 

PERSS2 

1 

-0.08781 

0.03358 

-2.62 

0.0099 

Third  level  Regression  Analysis  for  Exercises  1,2,3 


12:04  Wednesday,  December  1 

The  REG  Procedure 
Model:  M0DEL10 
Dependent  Variable:  TS23 
Analysis  of  Variance 


Sum  of 

Mean 

Source 

DF 

Squares 

Square 

F  Value 

Pr  >  F 

Model 

10 

8.38067 

0.83807 

14.72 

<.0001 

Error 

128 

7.28840 

0.05694 

Corrected  Total 

138 

15.66906 

Root  MSE 

0.23862 

R-Square 

0.5349 

Dependent 

Mean 

0.12950 

Adj  R-Sq 

0.4985 

Coeff  Var 

184.26947 

Parameter  Estimates 

Parameter 

Standard 

Variable 

DF 

Estimate 

Error 

t  Value 

Pr  >  1 1 1 

Intercept 

1 

-0.50056 

0.15867 

-3.15 

0.0020 

B0SIntel2 

1 

0.12448 

0.05538 

2.25 

0.0263 

B0SIntel3 

1 

0.20154 

0.05547 

3.63 

0.0004 

B0SMan2 

1 

-0.23144 

0.08071 

-2.87 

0.0048 

B0SFS9 

1 

-0.13123 

0.05672 

-2.31 

0.0223 

B0SMAS4 

1 

0.58920 

0.11780 

5.00 

<.0001 

B0SCC2 

1 

0.18261 

0.05731 

3.19 

0.0018 

MISCBattleTiming 

1 

-0.52596 

0.15863 

-3.32 

0.0012 

MISCInf oOps 

1 

0.26852 

0.11073 

2.43 

0.0167 

PERSS3RT0 

1 

0.35949 

0.06329 

5.68 

<.0001 

Humidity 

1 

0.58127 

0.18105 

3.21 

0.0017 

9 

,  2002 


10 

,  2002 
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Third  level  Regression  Analysis  for  Exercises  1,2,3  11 

12:04  Wednesday,  December  18,  2002 

The  REG  Procedure 
Model:  M0DEL11 
Dependent  Variable:  TA3 
Analysis  of  Variance 


Sum  of 

Mean 

Source 

DF 

Squares 

Square 

F  Value 

Pr  >  F 

Model 

10 

7.63123 

0.76312 

29.31 

<.0001 

Error 

128 

3.33280 

0.02604 

Corrected  Total 

138 

10.96403 

Root  MSE 

0.16136 

R-Square 

0.6960 

Dependent 

Mean 

0.08633 

Adj  R-Sq 

0.6723 

Coeff  Var 

186.91015 

Parameter  Estimates 

Parameter 

Standard 

Variable 

DF 

Estimate 

Error 

t  Value 

Pr  >  | t | 

Intercept 

1 

0.08980 

0 . 04943 

1  .82 

0.0716 

BOSManl 

1 

0.40832 

0.04661 

8.76 

<.0001 

B0SFS2 

1 

0.23808 

0.05374 

4.43 

<.0001 

B0SFS9 

1 

-0.14227 

0.04457 

-3.19 

0.0018 

B0SMAS1 

1 

-0.39912 

0.05216 

-7.65 

<.0001 

B0SCC1 

1 

-0.15818 

0.04280 

-3.70 

0.0003 

COMAFATDS 

1 

-0.73340 

0.12275 

-5.97 

<.0001 

COMMCS 

1 

0.49004 

0.10305 

4.76 

<.0001 

Staff Hud 

1 

-0.28177 

0.05568 

-5.06 

<.0001 

PERSS3RTO 

1 

0.11013 

0.03845 

2.86 

0.0049 

PERSALO 

1 

-0.09726 

0.03674 

-2.65 

0.0091 
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Third  level  Regression  Analysis  for  Exercises  1,2,3 


12:04  Wednesday,  December  1 


The  REG  Procedure 
Model:  M0DEL12 
Dependent  Variable:  TA21 
Analysis  of  Variance 


Sum  of 

Mean 

Source 

DF 

Squares 

Square 

F  Value 

Pr  >  F 

Model 

12 

16.40103 

1 .36675 

81 .35 

<.0001 

Error 

126 

2.11695 

0.01680 

Corrected  Total 

138 

18.51799 

Root  MSE 

0.12962 

R-Square 

0.8857 

Dependent 

Mean 

0.15827 

Adj  R-Sq 

0.8748 

Coeff  Var 

81 .89599 

Parameter  Estimates 


Variable  DF 
Intercept  1 
BOSIntell  1 
BOSManl  1 
B0SFS9  1 
B0SMAS7  1 
B0SCC1  1 
COMAFATDS  1 
MISCBattleTiming  1 
MISCBattleTempo  1 
MISCTOCActivity  1 
StaffHud  1 
PERSS3RTO  1 
PERSALO  1 


Parameter 

Standard 

Estimate 

Error 

-0.08608 

0.07210 

-0.28342 

0.05038 

-0.50502 

0.03914 

0.58484 

0.03961 

-0.19218 

0.03824 

0.10053 

0.05461 

-0.11419 

0.08325 

0.54202 

0.13319 

-0.46401 

0.10414 

-0.12301 

0.06071 

-0.11568 

0.04235 

0.37038 

0.04710 

-0.07024 

0.02956 

t  Value 

Pr  >  1 1 1 

-1 .19 

0.2348 

-5.63 

<.0001 

-12.90 

<.0001 

14.77 

<.0001 

-5.03 

<.0001 

1 .84 

0.0680 

-1.37 

0.1726 

4.07 

<.0001 

-4.46 

<.0001 

-2.03 

0 . 0449 

-2.73 

0.0072 

7.86 

<.0001 

-2.38 

0.0190 

12 

,  2002 
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Third  level  Regression  Analysis  for  Exercises  1,2,3  13 

12:04  Wednesday,  December  18,  2002 

The  REG  Procedure 
Model:  M0DEL13 
Dependent  Variable:  TA23 
Analysis  of  Variance 


Sum  of 

Mean 

Source 

DF 

Squares 

Square 

F  Value 

Pr  >  F 

Model 

16 

19.98794 

1  .24925 

86.23 

<.0001 

Error 

122 

1 .76745 

0.01449 

Corrected  Total 

138 

21 .75540 

Root  MSE 

0.12036 

R-Square 

0.9188 

Dependent 

Mean 

0.19424 

Adj  R-Sq 

0.9081 

Coeff  Var 

61 .96477 

Parameter  Estimates 

Parameter 

Standard 

Variable 

DF 

Estimate 

Error 

t  Value 

Pr  >  1 1 1 

Intercept 

1 

-0.12844 

0.15545 

-0.83 

0.4103 

BOSIntell 

1 

0.65869 

0.08078 

8.15 

<.0001 

B0SIntel2 

1 

0.61373 

0.06464 

9.49 

<.0001 

B0SIntel3 

1 

0.64366 

0.06899 

9.33 

<.0001 

B0SIntel4 

1 

0.79420 

0.08264 

9.61 

<.0001 

B0SFS9 

1 

0.21125 

0.03982 

5.30 

<.0001 

B0SMAS6 

1 

0.61269 

0.11133 

5.50 

<.0001 

B0SADA1 

1 

-0.98150 

0.21394 

-4.59 

<.0001 

B0SADA2 

1 

-1 .06330 

0.18544 

-5.73 

<.0001 

B0SCC1 

1 

0.24535 

0.06423 

3.82 

0.0002 

B0SCC2 

1 

0.03435 

0.03708 

0.93 

0.3561 

BRIntel 

1 

-0.70477 

0.08414 

-8.38 

<.0001 

MISCBattleTiming 

1 

0.65510 

0.13993 

4.68 

<.0001 

MOPPLevel 

1 

2.02343 

0.35583 

5.69 

<.0001 

Staff Hud 

1 

0.19638 

0.04040 

4.86 

<.0001 

PERSS3RTO 

1 

-0.19305 

0.05341 

-3.61 

0.0004 

Humidity 

1 

-0.24889 

0.10436 

-2.39 

0.0186 
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Third  level  Regression  Analysis  for  Exercises  1,2,3  14 

12:04  Wednesday,  December  18,  2002 

The  REG  Procedure 
Model:  M0DEL14 
Dependent  Variable:  Missionl 
Analysis  of  Variance 


Sum  of 

Mean 

Source 

DF 

Squares 

Square 

F  Value 

Pr  >  F 

Model 

7 

8.87069 

1 .26724 

26.14 

<.0001 

Error 

131 

6.35003 

0.04847 

Corrected  Total 

138 

15.22072 

Root  MSE 

0.22017 

R-Square 

0.5828 

Dependent 

Mean 

0.16835 

Adj  R-Sq 

0.5605 

Coeff  Var 

130.78294 

Parameter  Estimates 

Parameter 

Standard 

Variable 

DF 

Estimate 

Error 

t  Value 

Pr  >  1 1 1 

Intercept 

1 

0.31958 

0.05825 

5.49 

<.0001 

B0SMAS1 

1 

-0.37529 

0.07376 

-5.09 

<.0001 

B0SMAS2 

1 

0.53554 

0.06788 

7.89 

<.0001 

B0SMAS4 

1 

0.58174 

0.11083 

5.25 

<.0001 

B0SADA1 

1 

-0.26499 

0.07300 

-3.63 

0.0004 

COMMCS 

1 

0.46269 

0.13080 

3.54 

0.0006 

MISCBattleTiming 

1 

-0.89308 

0.12270 

-7.28 

<.0001 

PERSS3SGM 

1 

-0.11212 

0.05691 

-1.97 

0.0509 

Third  level  Regression  Analysis  for  Exercises  1,2,3  15 

12:04  Wednesday,  December  18,  2002 

The  REG  Procedure 
Model:  M0DEL15 
Dependent  Variable:  Mission2 
Analysis  of  Variance 


Sum  of 

Mean 

Source 

DF 

Squares 

Square 

F  Value 

Pr  >  F 

Model 

6 

3.48794 

0.58132 

30.12 

<.0001 

Error 

132 

2.54731 

0.01930 

Corrected  Total 

138 

6.03525 

Root  MSE 

0.13892 

R-Square 

0.5779 

Dependent 

Mean 

0.22158 

Adj  R-Sq 

0.5587 

Coeff  Var 

62.69290 

Parameter  Estimates 

Parameter 

Standard 

Variable 

DF 

Estimate 

Error 

t  Value 

Pr  >  | t | 

Intercept 

1 

0.21417 

0.03487 

6.14 

<.0001 

B0SFS2 

1 

-0.07909 

0.03552 

-2.23 

0.0277 

B0SMAS1 

1 

0.08428 

0.03821 

2.21 

0.0291 

B0SCC2 

1 

0.22173 

0.0341 1 

6.50 

<.0001 

B0SCC3 

1 

0.21830 

0.03727 

5.86 

<.0001 

MISCReconOps 

1 

-0.12503 

0.05306 

-2.36 

0.0199 

PERSS3NC0 

1 

-0.21450 

0.02813 

-7.62 

<.0001 
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Third  level  Regression  Analysis  for  Exercises  1,2,3 

12:04  Wednesday,  December  1 

The  REG  Procedure 
Model:  M0DEL16 
Dependent  Variable:  Mission3 
Analysis  of  Variance 


Sum  of 

Mean 

Source 

DF 

Squares 

Square 

F  Value 

Pr  >  F 

Model 

5 

2.79679 

0.55936 

18.24 

<.0001 

Error 

133 

4.07839 

0.03066 

Corrected  Total 

138 

6.87518 

Root  MSE 

0.17511 

R-Square 

0.4068 

Dependent 

Mean 

0.08417 

Adj  R-Sq 

0.3845 

Coeff  Var 

208.04027 

Parameter  Estimates 

Parameter 

Standard 

Variable 

DF 

Estimate 

Error 

t  Value 

Pr  >  1 1 1 

Intercept 

1 

-0.12607 

0.04416 

-2.85 

0.0050 

B0SIntel4 

1 

0.31680 

0.04177 

7.58 

<.0001 

B0SMAS4 

1 

-0.12874 

0.08502 

-1.51 

0.1324 

COMMCS 

1 

-0.19093 

0.10594 

-1.80 

0.0738 

MISCBattleTiming 

1 

0.39925 

0.10144 

3.94 

0.0001 

MOPPLevel 

1 

0.32704 

0.09351 

3.50 

0.0006 

Third  level  Regression  Analysis  for  Exercises  1,2,3 

12:04  Wednesday,  December  1 

The  REG  Procedure 
Model:  M0DEL17 

Dependent  Variable:  Mission5 
Analysis  of  Variance 


Sum  of 

Mean 

Source 

DF 

Squares 

Square 

F  Value 

Pr  >  F 

Model 

6 

3.87057 

0.64510 

33.54 

<.0001 

Error 

132 

2.53849 

0.01923 

Corrected  Total 

138 

6.40906 

Root  MSE 

0.13868 

R-Square 

0.6039 

Dependent 

Mean 

0.07050 

Adj  R-Sq 

0.5859 

Coeff  Var 

196.69330 

Parameter  Estimates 

Parameter 

Standard 

Variable 

DF 

Estimate 

Error 

t  Value 

Pr  >  1 1 1 

Intercept 

1 

-0.65542 

0.06045 

-10.84 

<.0001 

B0SIntel4 

1 

0.09927 

0.03492 

2.84 

0.0052 

BOSManl 

1 

0.33456 

0.04731 

7.07 

<.0001 

B0SADA2 

1 

0.38865 

0.04469 

8.70 

<.0001 

MISCBattleTiming 

1 

1 .27217 

0.11413 

11.15 

<.0001 

MOPPLevel 

1 

-0.72215 

0.11272 

-6.41 

<.0001 

PERSS3NC0 

1 

0.11363 

0.03259 

3.49 

0.0007 

16 

,  2002 


17 

,  2002 
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Appendix  M  -  Fourth  Linear  Regression  Run  To  Select  IV  For  Inclusion  In  DV  Models 


SAS  Code  (Pick  those  IV  that  had  a  significant p  value  (a  =  .05)  from  the  third  run): 

options  pageno=l  f ormdlim="-" ; 

TITLE  "Training  Data  123  Regression  Analysis'; 

DATA  Tngdatal23; 

INPUT  Dayl  Day2  Day3  Day4  Day5  Day6  Day7  Day8  Day9  DaylO  Dayll  Dayl2  Dayl3  Dayl4  Dayl5  Dayl6 
Dayl7  Dayl8  Dayl9  Day20  Hourl  Hour2  Hour3  Hour4  Hour5  Hour6  Hour7  Hour8  Hour9  HourlO  Hourll 
Hourl2  Hourl3  Hourl4  Hourl5  Hourl6  Hourl7  Hourl8  Hourl9  Hour20  Hour21  Hour22  Hour23  Hour24 
TOCAn  TOCDg  TOCBn  TOCBde  ExTypl  ExTyp2  ExTyp3  ExTyp4  ExNol  ExNo2  ExNo3  ExNo4  ExNo5  ExNo6  ExNo7 
ExNo8  ExNo9  ExNolO  ExNoll  ExNol2  ExNol3  ExNol4  ExNol5  ExNol6  ExNol7  ExNol8  ExNol9  ExNo20  SimNo 
SimJANS  SimJAST  SimCBS  SimCCTT  SimSTRM  BOSIntell  BOSIntel2  BOSIntel3  BOSIntel4  BOSManl  BOSMan2 
BOSFS1  BOSFS2  BOSFS3  BOSFS4  BOSFS5  BOSFS6  BOSFS7  BOSFS8  BOSFS9  BOSMAS1  BOSMAS2  BOSMAS3  BOSMAS4 
BOSMAS5  BOSMAS6  BOSMAS7  BOSADA1  BOSADA2  BOSCC1  BOSCC2  BOSCC3  BRIntel  BRMan  BRFS  BRMAS  BRADA 
BRCSS  BRC2  COMAFATDS  COMAMPS  COMASAS  COMCTIS  COMCSSCS  COMFAADC2  COMFBCB2  COMIMETS  COMMCS  COMAMDWS 
COMJSTARS  COMEPLARS  INTELTUAV  INTELJSTARS  INTELGSR  MISCBattleTiming  MISCBattleTempo 
MISCReconOps  MISCInfoOps  MISCTactics  MISCTOCActivity  MOPPLevel  StaffHud  PERSCMDR  PERSXO  PERSBC 
PERSS3  PERSS3NCO  PERSS3RTO  PERSS2  PERSFSO  PERSENGR  PERSALO  PERSS3SGM  SKYCLR  SKYOC  SKYRAIN 
WINDNONE  WINDLOW  WINDMOD  WINDHIGH  LightBG  LightFG  Noise  TEMPDB  TEMPWB  Humidity  TP1  TP2  TP3  TP4 
TP 5  TP 6  TP 7  TP 8  TP 9  TP10  TP11  TP12  TP13  TP14  TP15  TP16  TP17  TP18  TP19  TP20  TP21  TP22  TP23  TS1 
TS2  TS3  TS4  TS5  TS6  TS7  TS8  TS9  TS10  TS11  TS12  TS13  TS14  TS15  TS16  TS17  TS18  TS19  TS20  TS21 
TS22  TS23  TA1  TA2  TA3  TA4  TA5  TA6  TA7  TA8  TA9  TA10  TA11  TA12  TA13  TA14  TA15  TA16  TA17  TA18 
TA19  TA20  TA21  TA22  TA23  Missionl  Mission2  Mission3  Mission4  Mission5  Mission6 


CARDS; 
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0  0  0 

0 

0 

0 

0 

0 

0 

1 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0  0 
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0  0 
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0 

0 

1 

0 

0 
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0 
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0 

0 

0 

0 

0 
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0 

0 
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0 

0 

0 

0 

1 

0 
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0.0 

0.4 

0.0 

0.0 

0.0 

0.0 

*  Partial  Tng  Regression  Analysis; 

*  Make  regression  run  with  input/independent  variables  remaining  that  are  significant  after  full 
regression  run 

*  to  try  to  maximize  the  adjusted  r**2  showing  the  amount  of  the  regression  accounted  for; 
options  pageno=l  f ormdlim="-" ; 

proc  reg  data=Tngdatal23; 

TITLE  "Fourth  level  Regression  Analysis  for  Exercises  1,2,3"; 
model  TP10=BOSIntel4  BOSMAS2  BOSCC2  MOPPLevel; 

model  TP1 l=BOSIntel3  BOSCC1  BOSCC2  BOSCC3  BRIntel  StaffHud  PERSXO  PERSS3RTO  PERSS3SGM; 

model  TP12=BOSFS2  BOSFS7  BOSFS9  BOSMAS6  BOSCC1  COMAFATDS  MISCBattleTiming  PERSALO  Humidity; 

model  TP15=BOSIntel4  BOSFS2  MISCBattleTiming  PERSXO  ; 

model  TP21=BOSFS9  BOSMAS5  MISCBattleTempo  MISCInfoOps  ; 

model  TP2 3=BOSMAS 6  BRIntel  BRMAS  StaffHud; 

model  TS12=BOSManl  BOSFS9  BOSMAS1; 

model  TS18=BOSFS2  BOSMAS6  BOSCC1  PERSXO  ; 

model  TS19=BOSMAS6  PERSS2 ; 

model  TS23=BOSIntel2  BOSIntel3  BOSMan2  BOSFS9  BOSMAS4  BOSCC2  MISCBattleTiming  MISCInfoOps 
PERSS3RTO  Humidity; 

model  TA3=BOSManl  BOSFS2  BOSFS9  BOSMAS1  BOSCC1  COMAFATDS  COMMCS  StaffHud  PERSS3RTO  PERSALO; 
model  TA2 l=BOSIntell  BOSManl  BOSFS9  BOSMAS7  BOSCC1  MISCBattleTiming  MISCBattleTempo 
MISCTOCActivity  StaffHud  PERSS3RTO  PERSALO; 

model  TA23=BOSIntell  BOSIntel2  BOSIntel3  BOSIntel4  BOSFS9  BOSMAS6  BOSADA1  BOSADA2  BOSCC1 
BRIntel  MISCBattleTiming  MOPPLevel  StaffHud  PERSS3RTO  Humidity; 
model  Missionl=BOSMASl  BOSMAS2  BOSMAS4  BOSADA1  COMMCS  MISCBattleTiming  ; 
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model  Mission2=BOSFS2  B0SMAS1  B0SCC2  BOSCC3  MISCReconOps  PERSS3NC0  ; 
model  Mission3=BOSIntel4  MISCBattleTiming  MOPPLevel  ; 

model  Mission5=BOSIntel4  BOSManl  B0SADA2  MISCBattleTiming  MOPPLevel  PERSS3NCO 

run; 
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SAS  Output: 


Fourth  level  Regression  Analysis  for  Exercises  1,2,3  1 

12:04  Wednesday,  December  18,  2002 

The  REG  Procedure 
Model:  M0DEL1 
Dependent  Variable:  TP10 
Analysis  of  Variance 


Sum  of 

Mean 

Source 

DF 

Squares 

Square 

F  Value 

Pr  >  F 

Model 

4 

7.48707 

1 .87177 

42.55 

<.0001 

Error 

134 

5.89423 

0.04399 

Corrected  Total 

138 

13.38129 

Root  MSE 

0.20973 

R-Square 

0.5595 

Dependent 

Mean 

0.10791 

Adj  R-Sq 

0 . 5464 

Coeff  Var 

194.34998 

Parameter  Estimates 

Parameter 

Standard 

Variable 

DF 

Estimate 

Error 

t  Value 

Pr  >  1 1 1 

Intercept 

1 

0.09781 

0.03094 

3.16 

0.0019 

B0SIntel4 

1 

0.37701 

0.04647 

8.11 

<.0001 

B0SMAS2 

1 

0.31685 

0.05410 

5.86 

<.0001 

B0SCC2 

1 

-0.19909 

0.03768 

-5.28 

<.0001 

MOPPLevel 

1 

-0.46896 

0.11340 

-4.14 

<.0001 

Fourth  level  Regression  Analysis  for  Exercises  1,2,3  2 

12:04  Wednesday,  December  18,  2002 

The  REG  Procedure 
Model:  M0DEL2 
Dependent  Variable:  TP11 
Analysis  of  Variance 


Sum  of 

Mean 

Source 

DF 

Squares 

Square 

F  Value 

Pr  >  1 

Model 

9 

22.29580 

2.47731 

27.07 

<.000 

Error 

129 

1 1 .80492 

0.09151 

Corrected  Total 

138 

34.10072 

Root  MSE 

0.30251 

R-Square 

0.6538 

Dependent 

Mean 

0.43165 

Adj  R-Sq 

0.6297 

Coeff  Var 

70.08099 

Parameter  Estimates 

Parameter 

Standard 

Variable 

DF 

Estimate 

Error 

t  Value 

Pr  >  1 1 1 

Intercept 

1 

-0.26097 

0.24789 

-1.05 

0.2944 

B0SIntel3 

1 

0.27295 

0.05763 

4.74 

<.0001 

B0SCC1 

1 

0.85678 

0.18988 

4.51 

<.0001 

B0SCC2 

1 

0.85483 

0 . 1 9643 

4.35 

<.0001 

B0SCC3 

1 

1 .15989 

0.20510 

5.66 

<.0001 

BRIntel 

1 

-0.61238 

0.15375 

-3.98 

0.0001 

Staff Hud 

1 

-0.39140 

0.09395 

-4.17 

<.0001 

PERSXO 

1 

-0.27882 

0.06923 

-4.03 

<.0001 

PERSS3RT0 

1 

0.58359 

0.07289 

8.01 

<.0001 

PERSS3SGM 

1 

-0.38158 

0.07344 

-5.20 

<.0001 
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Fourth  level  Regression  Analysis  for  Exercises  1,2,3 

12:04  Wednesday,  December  1 

The  REG  Procedure 
Model:  M0DEL3 
Dependent  Variable:  TP12 
Analysis  of  Variance 


Sum  of 

Mean 

Source 

DF 

Squares 

Square 

F  Value 

Pr  >  F 

Model 

9 

6.48731 

0.72081 

20.77 

<.0001 

Error 

129 

4.47672 

0.03470 

Corrected  Total 

138 

10.96403 

Root  MSE 

0.18629 

R-Square 

0.5917 

Dependent 

Mean 

0.08633 

Adj  R-Sq 

0.5632 

Coeff  Var 

215.78368 

Parameter  Estimates 

Parameter 

Standard 

Variable 

DF 

Estimate 

Error 

t  Value 

Pr  >  1 1 1 

Intercept 

1 

0.97880 

0.09492 

10.31 

<.0001 

B0SFS2 

1 

-0.24508 

0.06243 

-3.93 

0.0001 

B0SFS7 

1 

0.33197 

0.09588 

3.46 

0.0007 

B0SFS9 

1 

-0.13939 

0.03816 

-3.65 

0.0004 

B0SMAS6 

1 

-0.37014 

0.08923 

-4.15 

<.0001 

B0SCC1 

1 

-0.28352 

0.06326 

-4.48 

<.0001 

COMAFATDS 

1 

0.52071 

0.11754 

4.43 

<.0001 

MISCBattleTiming 

1 

-1 .10636 

0.14265 

-7.76 

<.0001 

PERSALO 

1 

-0.16609 

0.04051 

-4.10 

<.0001 

Humidity 

1 

-0.48733 

0.13833 

-3.52 

0.0006 

Fourth  level  Regression  Analysis  for  Exercises  1,2,3 

12:04  Wednesday,  December  1 

The  REG  Procedure 
Model:  M0DEL4 
Dependent  Variable:  TP15 
Analysis  of  Variance 


Sum  of 

Mean 

Source 

DF 

Squares 

Square 

F  Value 

Pr  >  F 

Model 

4 

6.10327 

1 .52582 

35.99 

<.0001 

Error 

134 

5.68090 

0.04239 

Corrected  Total 

138 

1 1 .78417 

Root  MSE 

0.20590 

R-Square 

0.5179 

Dependent 

Mean 

0.09353 

Adj  R-Sq 

0.5035 

Coeff  Var 

220.15451 

Parameter  Estimates 

Parameter 

Standard 

Variable 

DF 

Estimate 

Error 

t  Value 

Pr  >  1 1 1 

Intercept 

1 

-0.25933 

0.05076 

-5.11 

<.0001 

B0SIntel4 

1 

0.32348 

0.04700 

6.88 

<.0001 

B0SFS2 

1 

0.21054 

0.04971 

4.24 

<.0001 

MISCBattleTiming 

1 

0.49106 

0.10598 

4.63 

<.0001 

PERSXO 

1 

0.13550 

0.04029 

3.36 

0.0010 

3 

,  2002 


4 

,  2002 
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Fourth  level  Regression  Analysis  for  Exercises  1,2,3 

12:04  Wednesday,  December  1 

The  REG  Procedure 
Model:  MODELS 
Dependent  Variable:  TP21 
Analysis  of  Variance 


Sum  of 

Mean 

Source 

DF 

Squares 

Square 

F  Value 

Pr  >  F 

Model 

4 

14.17579 

3.54395 

41 .27 

<.0001 

Error 

134 

11 .50766 

0.08588 

Corrected  Total 

138 

25.68345 

Root  MSE 

0.29305 

R-Square 

0.5519 

Dependent 

Mean 

0.24460 

Adj  R-Sq 

0.5386 

Coeff  Var 

119.80558 

Parameter  Estimates 

Parameter 

Standard 

Variable 

DF 

Estimate 

Error 

t  Value 

Pr  >  1 1 1 

Intercept 

1 

0.47554 

0.07136 

6.66 

<.0001 

B0SFS9 

1 

-0.32588 

0.05679 

-5.74 

<.0001 

B0SMAS5 

1 

-0.12732 

0.05531 

-2.30 

0.0229 

MISCBattleTempo 

1 

-0.97917 

0.13160 

-7.44 

<.0001 

MISCInf oOps 

1 

0.45966 

0.13416 

3.43 

0.0008 

Fourth  level  Regression  Analysis  for  Exercises  1,2,3 

12:04  Wednesday,  December  1 

The  REG  Procedure 
Model:  M0DEL6 
Dependent  Variable:  TP23 
Analysis  of  Variance 


Sum  of 

Mean 

Source 

DF 

Squares 

Square 

F  Value 

Pr  >  F 

Model 

4 

9.11062 

2.27766 

71  .47 

<.0001 

Error 

134 

4.27067 

0.03187 

Corrected  Total 

138 

13.38129 

Root  MSE 

0.17852 

R-Square 

0.6808 

Dependent 

Mean 

0.10791 

Adj  R-Sq 

0.6713 

Coeff  Var 

165.43190 

Parameter  Estimates 

Parameter 

Standard 

Variable 

DF 

Estimate 

Error 

t  Value 

Pr  >  1 1 1 

Intercept 

1 

0.39034 

0.11841 

3.30 

0.0013 

B0SMAS6 

1 

1  .01 120 

0.06566 

15.40 

<.0001 

BRIntel 

1 

0.31699 

0.08691 

3.65 

0.0004 

BRMAS 

1 

-0.71853 

0.11348 

-6.33 

<.0001 

Staff Hud 

1 

0.20794 

0.05561 

3.74 

0.0003 

5 

,  2002 


6 

,  2002 
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Fourth  level  Regression  Analysis  for  Exercises  1,2,3 

12:04  Wednesday,  December  1 

The  REG  Procedure 
Model:  M0DEL7 
Dependent  Variable:  TS12 
Analysis  of  Variance 


Sum  of 

Mean 

Source 

DF 

Squares 

Square 

F  Value 

Pr  >  F 

Model 

3 

12.02977 

4.00992 

52.40 

<.0001 

Error 

135 

10.32994 

0.07652 

Corrected  Total 

138 

22.35971 

Root  MSE 

0.27662 

R-Square 

0.5380 

Dependent 

Mean 

0.20144 

Adj  R-Sq 

0.5277 

Coeff  Var 

137.32159 

Parameter  Estimates 

Parameter 

Standard 

Variable 

DF 

Estimate 

Error 

t  Value 

Pr  >  1 1 1 

Intercept 

1 

0.04119 

0.03426 

1  .20 

0.2313 

BOSManl 

1 

-0.46001 

0.06273 

-7.33 

<.0001 

B0SFS9 

1 

0.73140 

0.05962 

12.27 

<.0001 

B0SMAS1 

1 

-0.28259 

0.07846 

-3.60 

0.0004 

Fourth  level  Regression  Analysis  for  Exercises  1,2,3 

12:04  Wednesday,  December  1 

The  REG  Procedure 
Model:  M0DEL8 
Dependent  Variable:  TS18 
Analysis  of  Variance 


Sum  of 

Mean 

Source 

DF 

Squares 

Square 

F  Value 

Pr  >  F 

Model 

4 

2.78871 

0.69718 

9.53 

<.0001 

Error 

134 

9.80122 

0.07314 

Corrected  Total 

138 

12.58993 

Root  MSE 

0.27045 

R-Square 

0.2215 

Dependent 

Mean 

0.10072 

Adj  R-Sq 

0.1983 

Coeff  Var 

268.51866 

Parameter  Estimates 

Parameter 

Standard 

Variable 

DF 

Estimate 

Error 

t  Value 

Pr  >  1 1 1 

Intercept 

1 

-0.02089 

0.03196 

-0.65 

0.5146 

B0SFS2 

1 

0.15970 

0.06463 

2.47 

0.0147 

B0SMAS6 

1 

-0.34669 

0.11461 

-3.02 

0.0030 

B0SCC1 

1 

0.15473 

0.06615 

2.34 

0.0208 

PERSXO 

1 

0.24325 

0.05060 

4.81 

<.0001 

7 

,  2002 


8 

,  2002 
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Fourth  level  Regression  Analysis  for  Exercises  1,2,3 

12:04  Wednesday,  December  1 

The  REG  Procedure 
Model:  M0DEL9 
Dependent  Variable:  TS19 
Analysis  of  Variance 


Sum  of 

Mean 

Source 

DF 

Squares 

Square 

F  Value 

Pr  >  F 

Model 

2 

7.27184 

3.63592 

133.93 

<.0001 

Error 

136 

3.69219 

0.02715 

Corrected  Total 

138 

10.96403 

Root  MSE 

0.16477 

R-Square 

0.6632 

Dependent 

Mean 

0.08633 

Adj  R-Sq 

0.6583 

Coeff  Var 

190.85602 

Parameter  Estimates 

Parameter 

Standard 

Variable 

DF 

Estimate 

Error 

t  Value 

Pr  >  1 1 1 

Intercept 

1 

0.09891 

0.02985 

3.31 

0.0012 

B0SMAS6 

1 

0.96695 

0.06001 

16.11 

<.0001 

PERSS2 

1 

-0.08781 

0.03358 

-2.62 

0.0099 

Fourth  level  Regression  Analysis  for  Exercises  1,2,3 


12:04  Wednesday,  December  1 

The  REG  Procedure 
Model:  M0DEL10 
Dependent  Variable:  TS23 
Analysis  of  Variance 


Sum  of 

Mean 

Source 

DF 

Squares 

Square 

F  Value 

Pr  >  F 

Model 

10 

8.38067 

0.83807 

14.72 

<.0001 

Error 

128 

7.28840 

0.05694 

Corrected  Total 

138 

15.66906 

Root  MSE 

0.23862 

R-Square 

0.5349 

Dependent 

Mean 

0.12950 

Adj  R-Sq 

0.4985 

Coeff  Var 

184.26947 

Parameter  Estimates 

Parameter 

Standard 

Variable 

DF 

Estimate 

Error 

t  Value 

Pr  >  1 1 1 

Intercept 

1 

-0.50056 

0.15867 

-3.15 

0.0020 

B0SIntel2 

1 

0.12448 

0.05538 

2.25 

0.0263 

B0SIntel3 

1 

0.20154 

0.05547 

3.63 

0.0004 

B0SMan2 

1 

-0.23144 

0.08071 

-2.87 

0.0048 

B0SFS9 

1 

-0.13123 

0.05672 

-2.31 

0.0223 

B0SMAS4 

1 

0.58920 

0.11780 

5.00 

<.0001 

B0SCC2 

1 

0.18261 

0.05731 

3.19 

0.0018 

MISCBattleTiming 

1 

-0.52596 

0.15863 

-3.32 

0.0012 

MISCInf oOps 

1 

0.26852 

0.11073 

2.43 

0.0167 

PERSS3RT0 

1 

0.35949 

0.06329 

5.68 

<.0001 

Humidity 

1 

0.58127 

0.18105 

3.21 

0.0017 

9 

,  2002 


10 

,  2002 
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Fourth  level  Regression  Analysis  for  Exercises  1,2,3  11 

12:04  Wednesday,  December  18,  2002 

The  REG  Procedure 
Model:  M0DEL11 
Dependent  Variable:  TA3 
Analysis  of  Variance 


Sum  of 

Mean 

Source 

DF 

Squares 

Square 

F  Value 

Pr  >  F 

Model 

10 

7.63123 

0.76312 

29.31 

<.0001 

Error 

128 

3.33280 

0.02604 

Corrected  Total 

138 

10.96403 

Root  MSE 

0.16136 

R-Square 

0.6960 

Dependent 

Mean 

0.08633 

Adj  R-Sq 

0.6723 

Coeff  Var 

186.91015 

Parameter  Estimates 

Parameter 

Standard 

Variable 

DF 

Estimate 

Error 

t  Value 

Pr  >  | t | 

Intercept 

1 

0.08980 

0 . 04943 

1  .82 

0.0716 

BOSManl 

1 

0.40832 

0.04661 

8.76 

<.0001 

B0SFS2 

1 

0.23808 

0.05374 

4.43 

<.0001 

B0SFS9 

1 

-0.14227 

0.04457 

-3.19 

0.0018 

B0SMAS1 

1 

-0.39912 

0.05216 

-7.65 

<.0001 

B0SCC1 

1 

-0.15818 

0.04280 

-3.70 

0.0003 

COMAFATDS 

1 

-0.73340 

0.12275 

-5.97 

<.0001 

COMMCS 

1 

0.49004 

0.10305 

4.76 

<.0001 

Staff Hud 

1 

-0.28177 

0.05568 

-5.06 

<.0001 

PERSS3RTO 

1 

0.11013 

0.03845 

2.86 

0.0049 

PERSALO 

1 

-0.09726 

0.03674 

-2.65 

0.0091 
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Fourth  level  Regression  Analysis  for  Exercises  1,2,3 


12:04  Wednesday,  December  1 


The  REG  Procedure 
Model:  M0DEL12 
Dependent  Variable:  TA21 
Analysis  of  Variance 


Sum  of 

Mean 

Source 

DF 

Squares 

Square 

F  Value 

Pr  >  F 

Model 

1 1 

16.36942 

1 .48813 

87.96 

<.0001 

Error 

127 

2.14857 

0.01692 

Corrected  Total 

138 

18.51799 

Root  MSE 

0.13007 

R-Square 

0.8840 

Dependent 

Mean 

0.15827 

Adj  R-Sq 

0.8739 

Coeff  Var 

82.17974 

Parameter  Estimates 


Variable  DF 
Intercept  1 
BOSIntell  1 
BOSManl  1 
B0SFS9  1 
B0SMAS7  1 
B0SCC1  1 
MISCBattleTiming  1 
MISCBattleTempo  1 
MISCTOCActivity  1 
StaffHud  1 
PERSS3RTO  1 
PERSALO  1 


Parameter 

Standard 

Estimate 

Error 

-0.10147 

0.07147 

-0.26640 

0.04900 

-0.50849 

0.03920 

0.60112 

0.03792 

-0.20500 

0.03721 

0.08249 

0.05319 

0.51993 

0.13267 

-0.51344 

0.09804 

-0.11479 

0.06063 

-0.12006 

0.04237 

0.36281 

0.04694 

-0.06970 

0.02966 

t  Value 

Pr  >  1 1 1 

-1.42 

0.1581 

-5.44 

<.0001 

-12.97 

<.0001 

15.85 

<.0001 

-5.51 

<.0001 

1 .55 

0.1234 

3.92 

0.0001 

-5.24 

<.0001 

CO 

CD 

0.0606 

-2.83 

0.0054 

7.73 

<.0001 

-2.35 

0.0203 

12 

,  2002 
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Fourth  level  Regression  Analysis  for  Exercises  1,2,3  13 

12:04  Wednesday,  December  18,  2002 

The  REG  Procedure 
Model:  M0DEL13 
Dependent  Variable:  TA23 
Analysis  of  Variance 


Sum  of 

Mean 

Source 

DF 

Squares 

Square 

F  Value 

Pr  >  F 

Model 

15 

19.97551 

1 .33170 

92.03 

<.0001 

Error 

123 

1 .77988 

0.01447 

Corrected  Total 

138 

21 .75540 

Root  MSE 

0.12029 

R-Square 

0.9182 

Dependent 

Mean 

0.19424 

Adj  R-Sq 

0.9082 

Coeff  Var 

61 .92898 

Parameter  Estimates 

Parameter 

Standard 

Variable 

DF 

Estimate 

Error 

t  Value 

Pr  >  1 1 1 

Intercept 

1 

-0.06032 

0.13687 

-0.44 

0.6602 

BOSIntell 

1 

0.64616 

0.07959 

8.12 

<.0001 

B0SIntel2 

1 

0.59517 

0.06142 

9.69 

<.0001 

B0SIntel3 

1 

0.62386 

0.06555 

9.52 

<.0001 

B0SIntel4 

1 

0.76372 

0.07576 

10.08 

<.0001 

B0SFS9 

1 

0.20860 

0.03970 

5.25 

<.0001 

B0SMAS6 

1 

0.57366 

0.10299 

5.57 

<.0001 

B0SADA1 

1 

-0.90467 

0.19709 

-4.59 

<.0001 

B0SADA2 

1 

-1  .00819 

0.17553 

-5.74 

<.0001 

B0SCC1 

1 

0.21674 

0.05628 

3.85 

0.0002 

BRIntel 

1 

-0.69620 

0.08358 

-8.33 

<.0001 

MISCBattleTiming 

1 

0.59960 

0.12639 

4.74 

<.0001 

MOPPLevel 

1 

1 .92155 

0.33821 

5.68 

<.0001 

Staff Hud 

1 

0.19585 

0.04037 

4.85 

<.0001 

PERSS3RTO 

1 

-0.22018 

0.04464 

-4.93 

<.0001 

Humidity 

1 

-0.24981 

0.10429 

-2.40 

0.0181 
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Fourth  level  Regression  Analysis  for  Exercises  1,2,3 

12:04  Wednesday,  December  1 

The  REG  Procedure 
Model:  M0DEL14 
Dependent  Variable:  Missionl 
Analysis  of  Variance 


Sum  of 

Mean 

Source 

DF 

Squares 

Square 

F  Value 

Pr  >  F 

Model 

6 

8.68252 

1  .44709 

29.22 

<.0001 

Error 

132 

6.53820 

0.04953 

Corrected  Total 

138 

15.22072 

Root  MSE 

0.22256 

R-Square 

0.5704 

Dependent 

Mean 

0.16835 

Adj  R-Sq 

0.5509 

Coeff  Var 

132.20293 

Parameter  Estimates 

Parameter 

Standard 

Variable 

DF 

Estimate 

Error 

t  Value 

Pr  >  1 1 1 

Intercept 

1 

0.32216 

0.05886 

5.47 

<.0001 

B0SMAS1 

1 

-0.32372 

0.06971 

-4.64 

<.0001 

B0SMAS2 

1 

0.49275 

0.06501 

7.58 

<.0001 

B0SMAS4 

1 

0.47737 

0.09841 

4.85 

<.0001 

B0SADA1 

1 

-0.25757 

0.07369 

-3.50 

0.0006 

COMMCS 

1 

0.39495 

0.12757 

3.10 

0.0024 

MISCBattleTiming 

1 

-0.87756 

0.12378 

-7.09 

<.0001 

Fourth  level  Regression  Analysis  for  Exercises  1,2,3 

12:04  Wednesday,  December  1 

The  REG  Procedure 
Model:  M0DEL15 
Dependent  Variable:  Mission2 
Analysis  of  Variance 


Sum  of 

Mean 

ource 

DF 

Squares 

Square 

F  Value 

Pr  >  F 

odel 

6 

3.48794 

0.58132 

30.12 

<.0001 

rror 

132 

2.54731 

0.01930 

orrected  Total 

138 

6.03525 

Root  MSE 

0.13892 

R-Square 

0.5779 

Dependent 

Mean 

0.22158 

Adj  R-Sq 

0.5587 

Coeff  Var 

62.69290 

Parameter  Estimates 

Parameter 

Standard 

Variable 

DF 

Estimate 

Error 

t  Value 

Pr  >  1 1 1 

Intercept 

1 

0.21417 

0.03487 

6.14 

<.0001 

B0SFS2 

1 

-0.07909 

0.03552 

-2.23 

0.0277 

B0SMAS1 

1 

0.08428 

0.03821 

2.21 

0.0291 

B0SCC2 

1 

0.22173 

0.0341 1 

6.50 

<.0001 

B0SCC3 

1 

0.21830 

0.03727 

5.86 

<.0001 

MISCReconOps 

1 

-0.12503 

0.05306 

-2.36 

0.0199 

PERSS3NC0 

1 

-0.21450 

0.02813 

-7.62 

<.0001 

14 

,  2002 


15 

,  2002 
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Fourth  level  Regression  Analysis  for  Exercises  1,2,3 

12:04  Wednesday,  December  1 

The  REG  Procedure 
Model:  M0DEL16 
Dependent  Variable:  Mission3 
Analysis  of  Variance 


Sum  of 

Mean 

Source 

DF 

Squares 

Square 

F  Value 

Pr  >  F 

Model 

3 

2.66789 

0.88930 

28.53 

<.0001 

Error 

135 

4.20729 

0.03117 

Corrected  Total 

138 

6.87518 

Root  MSE 

0.17654 

R-Square 

0.3880 

Dependent 

Mean 

0.08417 

Adj  R-Sq 

0.3744 

Coeff  Var 

209.73138 

Parameter  Estimates 

Parameter 

Standard 

Variable 

DF 

Estimate 

Error 

t  Value 

Pr  >  1 1 1 

Intercept 

1 

-0.15252 

0.04232 

-3.60 

0.0004 

B0SIntel4 

1 

0.28484 

0.03719 

7.66 

<.0001 

MISCBattleTiming 

1 

0.33945 

0.08725 

3.89 

0.0002 

MOPPLevel 

1 

0.19870 

0.06969 

2.85 

0.0050 

Fourth  level  Regression  Analysis  for  Exercises  1,2,3 

12:04  Wednesday,  December  1 

The  REG  Procedure 
Model:  M0DEL17 

Dependent  Variable:  Mission5 
Analysis  of  Variance 


Sum  of 

Mean 

Source 

DF 

Squares 

Square 

F  Value 

Pr  >  F 

Model 

6 

3.87057 

0.64510 

33.54 

<.0001 

Error 

132 

2.53849 

0.01923 

Corrected  Total 

138 

6.40906 

Root  MSE 

0.13868 

R-Square 

0.6039 

Dependent 

Mean 

0.07050 

Adj  R-Sq 

0.5859 

Coeff  Var 

196.69330 

Parameter  Estimates 

Parameter 

Standard 

Variable 

DF 

Estimate 

Error 

t  Value 

Pr  >  1 1 1 

Intercept 

1 

-0.65542 

0.06045 

-10.84 

<.0001 

B0SIntel4 

1 

0.09927 

0.03492 

2.84 

0.0052 

BOSManl 

1 

0.33456 

0.04731 

7.07 

<.0001 

B0SADA2 

1 

0.38865 

0.04469 

8.70 

<.0001 

MISCBattleTiming 

1 

1 .27217 

0.11413 

11.15 

<.0001 

MOPPLevel 

1 

-0.72215 

0.11272 

-6.41 

<.0001 

PERSS3NC0 

1 

0.11363 

0.03259 

3.49 

0.0007 

16 

,  2002 


17 

,  2002 
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Appendix  N  -  Linear  Regression  Expressions  for  the  17  DV  Models 


#  1  -  Primary  Task  10.  Destruction,  capture,  or  bypass  of  enemy  force. 


Variable 

Intercept 

BOSIntel4 

BOSMAS2 

BOSCC2 

MOPPLevel 

TP10 

Beta  Weight 

0.09781 

0.37701 

0.31685 

-0.19909 

-0.46896 

#  2  -  Primary  Taskl  1.  Fixing  enemy  in  position. 


Variable 

Intercept 

BOSIntel3 

BOSCC1 

BOSCC2 

BOSCC3 

TP11 

Beta  Weight 

-0.26097 

0.27295 

0.85678 

0.85483 

1.15989 

Variable 

BRIntel 

StaffHud 

PERSXO 

PERSS3RTO 

Beta  Weight 

-0.61668 

-0.3914 

-0.27882 

0.58359 

Variable 

PERSS3SGM 

Beta  Weight 

-0.46158 

#  4  -  Primary  Taskl5.  Destruction  of  first  echelon  forces. 


Variable 

Intercept 

BOS  Intel4 

BOSFS2 

MISCBattle 

Timing 

PERS  XO 

TP15 

Beta  Weight 

-0.61941 

0.41048 

0.21054 

0.49106 

0.1355 

#  5  -  Primary  Task  21.  Peacekeeping  operations. 


Variable 

Intercept 

BOSFS9 

BOS  MAS 5 

MISC  Battle 
Tempo 

MISC  Info 
Ops 

TP21 

Beta  Weight 

0.47554 

-0.36188 

-0.12732 

-0.97917 

0.45966 

#  6  -  Primary  Task23.  Planning 


Variable 

Intercept 

BOS  MAS6 

BR  Intel 

BR  MAS 

StaffHud 

TP23 

Beta  Weight 

0.39034 

1.0112 

0.31699 

0.71853 

0.20794 

#  7  -  Secondary  Taskl2.  Synchronization  with  supporting  forces. 


Variable 

Intercept 

BOS  Manl 

BOS  FS9 

BOS  MAS1 

TS12 

Beta  Weight 

0.04119 

-0.46001 

0.7314 

-0.35619 

#  8  -  Secondary  Taskl 8.  Deception  activities. 

Variable 

Intercept 

BOS  FS2 

BOS  MAS6 

BOS  CC1 

PERS  XO 

TS18 

Beta  Weight 

-0.02089 

0.1597 

-0.34669 

0.15473 

0.61261 

#  9  -  Secondary  Taskl 9.  Rear  operations. 


Variable 

Intercept 

BOS  MAS6 

PERS  S2 

TS19 

Beta  Weight 

0.09891 

0.96695 

-0.08781 

#  10  -  Secondary  Task23.  Planning 


Variable 

Intercept 

BOS  Intel2 

BOS  Intel3 

BOS  Man2 

BOSFS9 

TS23 

Beta  Weight 

-0.50056 

0.12448 

0.20154 

-0.66144 

-0.1313 

Variable 

BOS  MAS4 

BOS  CC2 

MISC  Battle  Timing 

MISC  Info  Ops 

Beta  Weight 

0.5892 

0.18261 

-0.56196 

0.26861 

Variable 

PERS  S3RTO 

Humidity 

Beta  Weight 

0.35949 

0.58127 
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#11-  Tertiary  Task3.  Movement  to  the  line  of  departure. 


Variable 

Intercept 

BOS  Manl 

BOS  FS2 

BOS  FS9 

BOS  MAS1 

TA3 

Beta  Weight 

0.0898 

0.40832 

0.66808 

-0.14227 

-0.39912 

Variable 

BOS  CC1 

COM  AFATDS 

COM  MCS 

Staff  Hud 

Beta  Weight 

-0.15818 

-0.7414 

0.49004 

-0.35177 

Variable 

PERS  S3RTO 

PERS  AFO 

Beta  Weight 

0.11013 

-0.09726 

#  12  -  Tertiary  Task21.  Peacekeeping  operations. 


Variable 

Intercept 

BOS  Intell 

BOS  Manl 

BOS  FS9 

BOS  MAS7 

TA21 

Beta  Weight 

-0.10147 

-0.2664 

-0.50849 

0.60112 

-0.205 

Variable 

BOS  CC1 

MISC  Battle  Timing 

MISC  Battle 
Tempo 

MISC  TOC 
Activity 

Beta  Weight 

0.08249 

0.51993 

-0.51344 

-0.11479 

Variable 

Staff  Hud 

PERS  S3RTO 

Beta  Weight 

-0.12006 

0.36351 

#  13  -  Tertiary  Task23.  Planning 


Variable 

Intercept 

BOS  Intell 

BOS  Intel2 

BOS  Intel3 

BOS  Intel4 

TA23 

Beta  Weight 

-0.06032 

0.64616 

0.59517 

0.66686 

0.76372 

Variable 

BOS  FS9 

BOS  MAS6 

BOS  ADA1 

BOS  ADA2 

Beta  Weight 

0.2086 

0.66366 

-0.90467 

-1.00819 

Variable 

BOS  CC1 

BR  Intel 

MISC  Battle  Timing 

MOPP  Level 

Beta  Weight 

0.21674 

-0.6962 

0.5996 

1.92155 

Variable 

Staff  Hud 

PERS  S3RTO 

Humidity 

Beta  Weight 

0.19585 

-0.22018 

-0.24981 

#  14  -  Mission  1.  Pre  Operations  Planning 


Variable 

Intercept 

BOS  MAS1 

BOS  MAS2 

BOS  MAS4 

BOS  ADA1 

MISSIONl 

Beta  Weight 

0.32216 

-0.41072 

0.49275 

0.47737 

-0.61766 

Variable 

COM  MCS 

MISC  Battle  Timing 

Beta  Weight 

0.39495 

-0.87756 

#  15  -  Mission2.  Movement  to  Contact 
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-0.72215 
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Appendix  O  -  Description  of  the  CoHOST  Simulation. 

The  CoHOST  computer  simulation  models  used  in  this  dissertation  were  the  result  of  a 
previous  research  study  conducted  by  the  Anny.  Their  use  in  this  dissertation  represents  how 
continued  utilization  can  be  made  of  computer  simulations  developed  for  other  purposes. 
Considering  the  cost  and  time  required  to  develop  simulations  such  as  CoHOST,  their  use  in 
subsequent  efforts  like  COMPASS  represent  a  continued  payback  to  the  model  developers  in 
terms  of  utilization  and  effectiveness  of  the  models. 

Background. 

In  the  late  1990’s  the  U.  S.  Army  was  engaged  in  a  number  of  design  initiatives  for 
improving  the  operation  of  command  and  control  centers.  The  Human  Research  and 
Engineering  Directorate  of  the  U.S.  Anny  Research  Laboratory  (HRED-ARL)  was 
commissioned  to  develop  a  series  of  computer  simulations  to  replicate  existing  configurations 
and  to  investigate  potential  configurations  of  battalion  and  brigade  TOCs.  A  series  of  computer 
models  called  Computer  modeling  Of  Human  Operator  System  Tasks  (CoHOST)  (Middlebrooks 
et  ah,  1999b)  was  written  and  results  were  developed  that  addressed  some  of  the  questions  being 
posed  in  regard  to  the  modernization  of  TOCs  especially  in  regard  to  the  development  of  new 
digital  communications  systems  for  use  in  it.  These  new  digital  communications  systems  are  at 
the  heart  of  a  change  of  the  operational  paradigm  in  command  and  control  TOCs  that  the  Army 
is  now  undergoing.  Figure  55  illustrates  the  components  in  a  typical  Army  battalion  level  TOC. 
The  rectangles  represent  the  different  vehicles  in  the  unit.  Personnel  are  listed  inside  the  vehicle 
box  and  the  communication  systems  for  each  vehicle  are  listed  beside  it.  This  select  group  of  24 
people,  along  with  the  vehicles  and  communications  systems  they  use,  represent  those  people 
directly  concerned  with  battlespace  management  and  it  is  this  working  group  that  is  modeled  in 
CoHOST. 
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Figure  55  -  TOC  Diagram 
(Middlebrooks  et  al.,  1999b) 

This  project  addressed  the  ability  of  the  human  component  of  the  TOC  system  to  perform 
under  a  new  operational  paradigm.  As  communications  systems  are  passing  greater  and  more 
accurate  volumes  of  infonnation  in  real  time  the  question  to  be  asked  is  “can  the  soldier  absorb 
this  information  and  be  able  to  react  to  the  stream  of  data  being  presented  to  him/her  also  in  real 
time?”  Can  these  activities  be  performed  while  the  vehicle  is  moving  over  extended  distances 
and  during  extended  time  periods?  Do  the  combined  effects  of  fatigue,  noise,  and  vibration  that 
are  sustained  by  an  operator  cause  that  person  to  become  what  is  described  as  a  “cognitive 
causality”?  The  CoHOST  computer  models  and  project  looked  at  some  of  these  issues  and 
provided  findings  that  addressed  some  of  these  questions. 

Application  of  a  Taxonomy  of  Human  Performance. 

A  With  work  first  published  in  1954,  Edwin  Fleishman  (Fleishman,  1975)  began  what 
would  turn  into  a  lifetime  effort  focused  on  the  development  of  taxonomic  descriptors  of  work 
performance.  The  resulting  taxonomy  (Fleishman  and  Quaintance,  1984)  presents  a  set  of  skills 
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and  abilities  that  can  be  used  to  describe  human  performance  characteristics  in  any  general  work 
situation.  Fleishman  stated  (Fleishman,  1975;  Fleishman,  1978)  that  some  kind  of  taxonomy  of 
human  perfonnance  is  required  which  provides  an  integrative  framework  and  common  language 
applicable  to  a  variety  of  basic  and  applied  areas.  He  goes  on  further  to  state  that  it  does  appear 
that  predictions  and  generalizations  about  human  performance  may  be  enhanced  by  some  linkage 
of  task  classification  systems  based  on  human  abilities  and  task  characteristics.  In  1988 
Fleishman  (Fleishman,  1988)  quoted  earlier  1947  work  by  others  with  the  observation  that 
apparatus  tests  of  perceptual  motor  abilities  had  been  found  to  have  considerable  validity  for 
predicting  the  success  of  pilots  and  bombardiers  in  getting  through  training  during  World  War  II. 

Comments  by  others  point  out  that  Fleishman’s  work  tends  to  be  neglected  in  the 
mainstream  of  human  infonnation  processing  research,  perhaps  due  to  the  fact  that  the  skills  and 
abilities  in  the  taxonomy  are  only  based  on  factor  analyses  and  are  void  of  any  process 
description.  However,  the  tests  used  by  Fleishman  to  develop  the  taxonomy  belong  to  the  same 
type  of  performance  tests  that  are  studied  in  Wickens’  more  accepted  dual  task  experiments  and 
therefore  deserve  closer  scrutiny  (Sanders,  1997).  There  have  been  many  attempts  in  the  human 
factors  community  to  develop  similar  descriptions  of  human  performance  and  while  this 
taxonomy  may  not  be  generally  accepted  by  all  for  every  attempt  at  evaluations  of  human 
performance,  it  does  provide  a  set  of  skill  and  ability  descriptors  that  are  heavily  weighted  to 
cognitive  performance. 

Previous  work  at  the  U.S.  Army  Research  Laboratory  (ARL)  (Knapp,  1996a;  Knapp, 
1996b;  Knapp,  Johnson,  Barnette,  Wojciechowski,  Kilduff,  Bird,  and  Plott,  1997c;  Schipani  et 
ah,  1998),  and  the  U.S.  Army  Research  Institute  (ARI)  (Seven,  Akman,  Muckier,  Knapp,  and 
Burnstein,  1991)  identified  this  job  skill  and  ability  taxonomy  (Fleishman,  1984;  Fleishman  and 
Quaintance,  1984)  and  stated  that  it  showed  promise  to  provide  the  basis  for  workload  scaling  in 
Anny  battalion  level  command  and  control  modeling  efforts.  This  taxonomy  consists  of  52 
skills  and  abilities  that  include  mental  processing,  sensory  perception  and  fine  and  gross  motor 
skills.  The  selection  of  this  taxonomy  was  influenced  by  its  detailed  decomposition  of  mental 
abilities  and  the  existence  of  behaviorally  anchored  rating  scales  (Knapp  et  ah,  1997c). 
Subsequently,  50  of  the  52  skills  and  abilities  from  the  taxonomy  were  adopted  to  support  work 
that  was  performed  for  the  U.S.  Army  Intelligence  Center  at  Fort  Huachuca,  Arizona.  This  work 
sought  to  detennine  basic  soldier  training  requirements  needed  to  provide  requisite  skills  and 
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abilities  for  various  Military  Occupational  Specialty  (MOS)  at  the  Intelligence  Center’s  basic 
soldier  training  units.  As  shown  in  Figure  56,  the  taxonomy  was  grouped  into  eight  demand 
categories  of  reasoning,  speed-loaded,  conceptual,  communications,  visual,  auditory, 
psychomotor,  and  gross  motor.  Knapp  stated  that  (Knapp  et  ah,  1997c)  “  Each  skill  and  ability 
has  an  associated  behaviorally  anchored  rating  scale  that  ranges  from  “1”  for  a  very  low  level 
demand,  to  “7”  for  the  highest  demand.  Definitions  for  all  50  skills  and  abilities,  along  with  their 
behaviorally  anchored  scales,  is  documented  in  a  separate  review  of  this  taxonomy  (Seven  et  ah, 
1991).” 
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Figure  56  -  Fleishman’s  Knowledge,  Skills,  and  Abilities  Taxonomy 
(Middlebrooks  et  ah,  1999b) 


Using  Fleishman’s  taxonomy,  a  database  was  developed  using  questionnaires  using 
Likert  -  like  7  point  behaviorally  anchored  questions  and  was  administered  to  U.S.  Army  SMEs. 
This  questionnaire  associated  physical  and  mental  skills  and  abilities  from  the  taxonomy  to 
performance  tasks  such  as  “receive  and  record  a  radio  message”  that  operators  would  be 
expected  to  execute  in  the  performance  of  their  duties  in  a  TOC  during  the  conduct  of  battlefield 
operations.  This  database  then  provided  a  numerical  basis  for  a  computer  simulation  model  to 
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calculate  a  workload  estimate  for  each  individual  based  on  the  tasks  being  perfonned  at  the 
instant  of  the  calculation.  The  time  interval  selected  for  workload  calculation  updates  was  100 
seconds.  Resulting  from  this,  over  the  course  of  a  simulation  run,  a  profde  of  individual 
workload  and  utilization  rates  was  established  for  each  member  of  the  workgroup  at  a  100 
second  resolution.  The  data  was  captured  so  that  the  workload  rates  could  be  decomposed  into 
the  individual  elements  of  the  taxonomy  so  that  the  amount  of  time  spent  by  the  individual  in  the 
different  cognitive  and  physical  performance  categories  could  be  determined.  These  workload 
and  utilization  profiles  were  then  analyzed  following  the  simulation  run  using  multivariate 
statistical  techniques  to  predict  whether  individuals  became  cognitively  saturated  and  therefore 
unable  to  effectively  perform  their  assigned  tasks. 

Tactical  Scenario. 

The  tactical  mission  was  modeled  as  a  force-on-force  operation  occurring  over  several 
hours.  Different  scenarios  that  were  developed  include  the  phases  of  pre-operations  planning, 
movement-to-contact,  deliberate  defense,  and  hasty  attack.  Some  scenarios  reflect  heavy  combat 
actions  and  others  reflect  extended  movement  and  reconnaissance  type  operations  as  shown  in 
Figure  57.  A  model  input  file  consisting  of  scenario  voice  and  digital  messages  expected  to  be 
sent  to  and  from  the  TOC  during  the  course  of  the  tactical  mission  was  generated  using  training 
scenarios  for  Southwest  Asia  operations  and  OMS/MP  (Operations  Mission  Summary  /  Mission 
Profile)  movement  rates  as  provided  by  the  U.S.  Anny  Armor  Center  at  Fort  Knox,  Kentucky. 
The  input  file  indicates  the  time  each  message  occurs,  where  it  is  received  and  who  or  what 
equipment  receives  it,  and  the  subsequent  routing  and  task  flow  initiated  by  this  message.  Tasks 
performed  in  response  to  these  messages  come  from  an  external  source  (usually  a  radio,  digital 
link,  or  coworker),  and  are  labeled  “reactive”,  and  either  “voice”  or  ‘digital”.  In  addition  to 
external  messages,  the  scenario  file  also  contains  "internal  infonnation  messages’  that  are  mental 
“triggers’  for  personnel  to  periodically  perfonn  “proactive”  (self-initiated)  tasks  that  are  an 
essential  part  of  C2  operations  and  workstation  database  manipulation  (Knapp,  Johnson, 
Barnette,  Wojciechowski,  Kilduff,  and  Swoboda,  1997a).  Examples  of  these  proactive  tasks  are 
situation  assessment  checks,  updating  documentation  (plans,  orders,  etc.),  preparing  status 
reports,  and  calling  up  windows  of  infonnation  for  review. 
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Figure  57  -  Tactical  Scenario  Overlay 
(Middlebrooks  et  al.,  1999b) 


The  CoHOST  Computer  Simulation  Model. 

The  discrete  event  programming  language  Micro  Saint™  provided  the  CoHOST 
programming  environment  to  develop  the  protocols  and  conventions  to  input  the  tasks,  task 
sequences,  flow  logic,  and  task  timing  and  workload  data  from  the  network  diagrams  into  an 
executable  model.  The  computer  model  works  according  to  a  basic  “input-throughput-output” 
scheme  as  shown  in  Figure  58.  That  is,  the  inputs  to  the  model  are  message  events  from  the 
scenario  input  file,  which  present  an  information  event  stream  in  a  time  sequence  synchronized 
to  mission  activity  phases.  As  these  information  events  enter  the  model,  tasks  are  triggered  and 
performed  in  a  pattern  that  reflects  the  logic  for  task  branching,  interrupt  priorities,  time  outs, 
and  collaborative  (interactive)  tasks.  Any  information  event  that  triggers  a  staff  huddle  always 
has  the  highest  priority  (Knapp  et  al.,  1997a). 
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Figure  58  -  Conceptual  Model  of  TOC  Operations 
(Middlebrooks  et  al.,  1999b) 

CoHOST  runs  on  an  IBM-compatible  PC  running  Windows  2000  (or  higher).  During 
model  execution,  a  graphical  user  interface  (GUI)  screen  displays  the  progress  of  tasks  being 
performed  by  each  C2  section  and  individual  soldier  position,  as  information  messages  enter  the 
system.  Bar  and  pie  charts  on  the  GUI  display  allow  an  observer  to  get  an  initial  look  at  whether 
staff  sections  and  individuals  are  keeping  pace  or  falling  behind  in  their  information  processing, 
as  well  as  how  busy  or  idle  they  are  as  scenario  time  goes  on.  A  screen  print  from  this  real-time 
display  is  shown  in  Figure  59. 
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Figure  59  -  CoHOST  Model  Action  View  Display 
(Middlebrooks  et  al.,  1999b) 


The  model  was  developed  in  three  steps  that  occurred  iteratively  and  in  parallel: 

(1)  Cognitive  task  analysis  and  workload  measurement  for  battalion 
command  and  control  tasks,  using  techniques  from  the  most  recent  human  performance  and 
related  literature; 

(2)  Obtaining  and  translating  scenarios  and  task  flow  data  from  pertinent 
documentation  and  battalion  C2  SMEs; 

(3)  Exercising  the  Micro  Saint™  discrete  event  simulation  programming 
language  to  simulate  the  task  and  flow  data  from  steps  one  and  two.  Following  data  input,  the 
C2  computer  model  was  debugged  and  executed,  and  the  resulting  output  data  were  analyzed 
using  descriptive  and  comparative  statistics.  An  example  of  the  task  flow  logic  contained  in  one 
of  the  CoHOST  models  is  shown  in  Figure  60. 
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Heavy  Maneuver  Battalion  Command 
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Figure  60  -  CoHOST  Model  Network  Flow  Diagram 


Results  From  Original  CoHOST  Project. 

Each  CoHOST  model  was  executed  using  communication  messages  from  the  tactical 
scenario  as  driver  events  for  the  simulation.  The  dependent  measures  that  were  evaluated  were: 
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1)  Tasks  dropped  -  those  tasks  that  an  operator  did  not  complete  for  any  reason. 

2)  Tasks  interrupted  -  those  tasks  whose  performance  was  interrupted  by  another 
task  or  event  of  higher  priority. 

3)  Number  of  task  queues  generated  -  the  number  of  times  an  incoming  task  was 
assigned  to  a  queue  wait  state  because  the  operator  identified  to  perform  the  task 
was  busy  perfonning  another  task  of  equal  or  higher  priority. 

4)  Task  backlog  work  -  off  time  -  the  amount  of  time  it  took  for  an  operator  to 
eliminate  the  tasks  that  were  queued  up  for  execution. 


Additional  analyses  were  perfonned  to  assess  the  reasons  for  and  types  of  information  flow 
bottlenecks.  The  purpose  of  this  review  was  to  identify  why  tasks  got  dropped,  queued,  and /  or 
interrupted. 

Initially,  three  CoHOST  models  were  executed  with  varying  configurations  of 
organizational  configuration  and  implementation  of  digital  communications  equipment.  The 
results  are  summarized  in  Figure  61. 


Percentage  Workload  for  Eight  Skill  Ability  Categories  by  Soldier  Rank  Levels 


Baseline  Traditional  Integrated  Baseline  Traditional  Integrated  Baseline  Traditional 

Officers  Officers  Officers  NCO  NCOs  Enlisted  Jr.  Enlisted  Jr.  Enlisted 


Figure  61  -  Percent  of  Time  Spent  In  Each  Performance  Category  of  the  Taxonomy 

(Knapp  et  ah,  1997c) 

Figure  61  presents  results  from  the  three  runs  with  information  organized  according  to  the 
taxonomy.  The  3  groups  of  bars  represent  information  from  officers,  NCO’s  and  junior  enlisted 
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personnel.  The  3  bars  within  each  group  represent  the  data  from  each  run.  The  three  runs  were 
identified  as: 

1)  Baseline  Model  -  Personnel  and  equipment  configuration  according  to  the  then 
(1996)  mode  of  TOC  operation  with  analog  communications  equipment  (Knapp  et 
al„  1997a). 

2)  Traditional  Model  -  Same  as  the  baseline  model  but  with  first  generation  digital 
communications  equipment  (Knapp,  Johnson,  Barnette,  Wojciechowski,  Kilduff, 
and  Swoboda,  1997b). 

3)  Integrated  Model  -  Reorganized  personnel  organizational  structure  to  capitalize 
on  enhanced  communications  equipment  capabilities  and  objective  digital 
communications  equipment  (Knapp  et  ah,  1997c). 

There  was  no  data  for  the  junior  enlisted  for  the  Integrated  model  run  because  all  the  junior 
enlisted  personnel  were  eliminated  by  the  personnel  reorganization  for  that  model  run. 

The  sections  of  each  bar  graph  are  color  coded  to  correspond  to  the  eight  categories  of  the 
taxonomy  as  indicated  in  the  legend.  The  top  category  represented  the  amount  of  time  spent 
performing  the  proactive  think  -  ahead  reasoning  cognitive  tasks  that  are  critically  important  for 
situation  analysis  and  decision  making  abilities.  The  next  category  is  the  cognitive  speed  loaded 
category  that  is  indicative  of  activity  that  requires  immediate  attention  for  quick  servicing  of  the 
activity  before  the  content  of  the  activity  becomes  obsolete.  Looking  at  the  three  bar  graphs  for 
the  3  model  runs  for  the  officers  at  the  left  side  of  Figure  61,  it  can  be  seen  that  in  the  baseline 
model  the  officers  were  modeled  as  being  able  to  spend  about  10%  of  their  time  performing  the 
proactive  think  -  ahead  tasks  necessary  to  maintain  cognitive  awareness  of  the  battlefield  and 
develop  decisions  on  what  actions  to  take  next.  Subject  Matter  Expert  opinion  validated  that  this 
estimate  roughly  corresponded  to  the  circumstances  of  actual  battle.  The  middle  bar  from  the 
traditional  model  run  that  simulated  first  generation  digital  communications  equipment  for  the 
officers  show  that  this  activity  was  greatly  reduced  being  almost  totally  supplanted  by  the  speed 
loaded  activity  of  the  next  taxonomic  category.  The  third  run  from  the  integrated  model  that 
simulated  the  full  capabilities  of  digital  communications  equipment  being  developed  and  a 
reorganization  of  personnel  to  take  full  advantage  of  it  shows  an  even  worse  situation  with 
almost  all  reasoning  activity  disappeared. 

The  explanation  for  this  phenomenon  comes  from  a  realization  that  while  each  element  of 
the  communications  equipment  was  perfonning  exactly  as  it  was  designed,  the  design  was  based 
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on  maximizing  hardware  system  perfonnance  that  did  not  include  the  human  as  an  integral 
component  of  the  system.  The  result  was  that  increased  message  arrival  rates  coming  from 
enhanced  communications  systems  were  forcing  the  decision  makers  to  focus  their  attention  to 
just  trying  to  keep  up  with  and  react  to  the  messages  with  the  result  that  there  was  no  time  left  to 
analyze  and  interpret  the  information  they  were  receiving.  Thus,  in  this  series  of  simulation  runs 
the  officer  decision  makers  went  from  a  pseudo  proactive  think  -  ahead  reasoning  mode  to  an 
almost  total  reactive  speed  loaded  mode  while  trying  to  keep  up  with  the  increased  message 
traffic.  A  contributing  element  to  this  situation  was  the  elimination  of  the  junior  enlisted 
personnel  whose  primary  duties  were  to  function  as  equipment  operators.  With  the 
organizational  paradigm  associated  with  the  new  digital  communications  equipment,  the  officer 
decision  makers  were  required  to  sit  at  and  operate  their  own  communications  consoles  and  had 
to  personally  interact  with  the  incoming  message  traffic. 

Summary  of  Original  CoHOST  Project. 

By  looking  at  which  individuals  were  predicted  to  be  workload  saturated  for  each  model 
run  condition,  a  project  conclusion  was  reached  that  increasing  automation  does  not  necessarily 
improve  human  decision  making  performance  and  may,  in  fact,  degrade  it. 
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Appendix  P  -  DV  /  IV  Correlation  Matrices  for  all  Tasks 


Table  41  -  DV  /  IV  Correlation  Matrix  For  Each  Performance  Task 


#  1  -  Primary  TasklO.  Destruction,  capture,  or  bypass  of  enemy  force.  | 

Variable 

Intercept 

B0SIntel4 

B0SMAS2 

B0SCC2 

MOPPLevel 

All  IVs  1.0 

1 

1 

1 

1 

All  IVs  0.0 

0 

0 

0 

0 

DV  .25 

0 

0 

0 

0 

No  IVs 

DV  0.5 

1 

1 

1 

1 

BOSIntel4,  BOSMAS2,  BOSCC2,  MOPPLevel 

DV  .75 

1 

1 

1 

1 

BOSIntel4,  BOSMAS2,  BOSCC2,  MOPPLevel 

Beta  Weight 

0.09781 

0.37701 

0.31685 

-0.19909 

-0.46896 

#  2  -  Primary  Taskl  1 .  Fixing  enemy  in  position. 

Variable 

Intercept 

B0SIntel3 

B0SCC1 

B0SCC2 

B0SCC3 

BRlntel 

StaffHud 

PERSXO 

PERSS3RT0 

PERSS3SGM 

All  IVs  1.0 

1 

1 

1 

1 

1 

1 

1 

1 

1 

All  IVs  0.0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

DV  .25 

1 

0 

0 

0 

0 

1 

0 

1 

0 

DV  0.5 

0 

1 

1 

0 

0 

1 

1 

0 

0 

DV  .75 

0 

0 

0 

1 

0 

0 

1 

1 

0 

Beta  Weight 

-0.26097 

0.27295 

0.85678 

0.85483 

1 .15989 

-0.61668 

-0.3914 

-0.27882 

0.58359 

-0.46158 

BOSIntel3,  StaffHud,  PERSS3RTO 

BOSCC1,  BOSCC2,  StaffHud,  PERSXO 

BOSCC3,  PERSXO,  PERSS3RTO 

#  3  -  Primary  Taskl  2.  Synchronization  with  supporting  forces. 

Variable 

Intercept 

B0SFS2 

B0SFS7 

B0SFS9 

B0SMAS6 

B0SCC1 

COMAFATDS 

MISCBattle 

PERSALO 

Humidity 

All  IVs  1.0 

1 

1 

1 

1 

1 

1 

1 

1 

1 

All  IVs  0.0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

DV  .25 

1 

0 

0 

1 

1 

0 

0 

0 

1 

DV  0.5 

0 

1 

0 

1 

0 

0 

1 

0 

0 

DV  .75 

1 

0 

1 

0 

0 

1 

0 

1 

1 

Beta  Weight 

0.9788 

-0.24508 

0.41197 

-0.13939 

-0.37014 

-0.35361 

0.61071 

-1.10636 

-0.16609 

-0.48741 

BOSFS2,  BOSMAS6,  BOSCC1 ,  Humidity 

BOSFS7,  BOSMAS6,  MISCBattleTiming 

BOSFS2,  BOSFS9,  COMAFATDS,  PERSALO,  Humidity 
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#  4  -  Primary  Taskl  5.  Destruction  of  first  echelon  forces. 

Variable 

Intercept 

B0SIntel4 

B0SFS2 

MISCBattleTiming 

PERSXO 

All  IVs  1 .0 

1 

1 

1 

1 

All  IVs  0.0 

0 

0 

0 

0 

DV  .25 

0 

1 

-0.0600 

1 

BOSFS2,  MISCBattleTiming,  PERSXO 

DV  0.5 

1 

0 

0.43 

0 

BOSIntel4,  MISCBattleTiming,  PERSXO 

DV  .75 

1 

1 

0.36 

1 

BOSIntel4,  BOSFS2,  MISCBattleTiming,  PERSXO 

Beta  Weight 

-0.61941 

0.41048 

0.21054 

0.49106 

0.1355 

#  5  -  Primary  Task21 .  Peacekeeping  operations. 

Variable 

Intercept 

B0SFS9 

B0SMAS5 

MISCBattleTempo 

MISCInf oOps 

All  IVs  1.0 

1 

1 

1 

1 

All  IVs  0.0 

0 

0 

0 

0 

DV  .25 

0 

0 

1 

0 

MISCBattleTempo 

DV  0.5 

0 

0 

1 

1 

MISCBattleTempo,  MISCInfoOps 

DV  .75 

1 

1 

0 

1 

BOSFS9,  BOSMAS5,  MISCInfoOps 

Beta  Weight 

0.47554 

-0.36188 

-0.12732 

-0.97917 

0.45966 

#  6  -  Primary  Task23.  Planning 

Variable 

Intercept 

B0SMAS6 

BRIntel 

BRMAS 

StaffHud 

All  IVs  1.0 

1 

1 

1 

1 

All  IVs  0.0 

0 

0 

0 

0 

DV  .25 

0 

1 

0 

1 

BRIntel,  StaffHud 

DV  0.5 

1 

0 

0 

1 

BOSMAS6,  StaffHud 

DV  .75 

1 

1 

0 

1 

BOSMAS6,  BRIntel,  StaffHud 

Beta  Weight 

0.39034 

1  .0112 

0.31699 

0.71853 

0.20794 

#  7  -  Secondary  Taskl  2.  Synchronization  with  supporting  forces. 

Variable 

Intercept 

BOSManl 

B0SFS9 

B0SMAS1 

All  IVs  1.0 

1 

1 

1 

All  IVs  0.0 

0 

0 

0 

DV  .25 

1 

1 

1 

BOSManl,  BOSFS9,  BOSMAS1 

DV  0.5 

1 

1 

1 

BOSManl,  BOSFS9,  BOSMAS1 

DV  .75 

1 

1 

1 

BOSManl,  BOSFS9,  BOSMAS1 

Beta  Weight 

0.04119 

-0.46001 

0.7314 

-0.35619 

#  8  -  Secondary  Taskl  8.  Deception  activities. 

Variable 

Intercept 

B0SFS2 

B0SMAS6 

B0SCC1 

PERSXO 

All  IVs  1.0 

1 

1 

1 

1 

All  IVs  0.0 

0 

0 

0 

0 

DV  .25 

0 

0 

1 

0 

BOSCC1 

DV  0.5 

1 

0 

1 

0 

BOSFS2,  BOSCC1 

DV  .75 

1 

1 

0 

1 

BOSFS2,  BOSMAS6,  PERSXO 

Beta  Weight 

-0.02089 

0.1597 

-0.34669 

0.15473 

0.61261 

#  9  -  Secondary  T askl  9.  Rear  operations. 

Variable 

Intercept 

B0SMAS6 

PERSS2 

All  IVs  1.0 

1 

1 

All  IVs  0.0 

0 

0 

DV  .25 

1 

1 

BOSMAS6,  PERSS2 

DV  0.5 

1 

1 

BOSMAS6,  PERSS2 

DV  .75 

1 

1 

BOSMAS6,  PERSS2 

Beta  Weight 

0.09891 

0.96695 

-0.08781 
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#10-  Secondary  T ask2C 

5.  Planning 

Variable 

Intercept 

B0SIntel2 

B0Slntel3 

B0SMan2 

B0SFS9 

B0SMAS4 

B0SCC2 

All  IVs  1.0 

1 

1 

1 

1 

1 

1 

All  IVs  0.0 

0 

0 

0 

0 

0 

0 

DV  .25 

1 

1 

0 

1 

0 

0 

DV  0.5 

0 

0 

0 

0 

1 

0 

DV  .75 

0 

0 

0 

1 

1 

0 

Beta  Weight 

-0.50056 

0.12448 

0.20154 

-0.66144 

-0.1313 

0.5892 

0.18261 

MISCBattleTiming 

1 

0 

_ 0 _ 

_ 0 _ 

_ 0 _ 

-0.56196 


MISCInfoOps  PERSS3RT0  Humidity 
1  1  1 

0  0  0 

0 _ 0 _ 0 

0 _ 0 _ 0 

1 _ 0 _ 0 

0.26861  0.35949  0.58127 


BOSIntel2,  BOSIntel3,  BOSFS9 
BOSMAS4 

BOSFS9,  BOSMAS4,  MISCInfoOps 


Variable 
All  IVs  1.0 
All  IVs  0.0 
DV  .25 
DV0.5 
DV  .75 
Beta  Weight 


#11-  Tertiary  Task3.  Movement  to  the  line  of  departure. 


Intercept  BOSManl 
1 


B0SFS2 

1 

0 

0 

0 

1 


B0SFS9 

1 

0 

1 

0 

1 


B0SMAS1 

1 

0 

0 

0 

1 


B0SCC1 

1 

0 

1 

0 

1 


0.0898 


0.40832  0.66808  -0.14227  -0.39912  -0.15818 


C0MAFATDS 

1 

0 

0 

0 

0 

-0.7414 


C0MMCS 

1 

0 

0 

0 

0 

0.49004 


Staff Hud  PERSS3RT0  PERSAL0 
1  1  1 

0  0  0 

0 _ 1 _ 0 

0 _ 0 _ 0 

0 _ 1 _ 0 

-0.35177  0.11013  -0.09726 


BOSManl,  BOSFS9,  BOSCC1 ,  PERSS3RTO _ 

BOSManl 

BOSManl,  BOSFS2,  BOSFS9,  BOSMAS1 ,  BOSCC1 ,  PERSS3RTO 


#  12  -  Tertiary  Task21 .  F 

3eacekeeping  operations. 

Variable 

Intercept 

BOSIntell 

BOSManl 

B0SFS9 

B0SMAS7 

B0SCC1 

MISCBattleTiming 

MISCBattleTempo 

All  IVs  1.0 

1 

1 

1 

1 

1 

1 

1 

All  IVs  0.0 

0 

0 

0 

0 

0 

0 

0 

DV  .25 

0 

0 

0 

0 

0 

0 

0 

DV  0.5 

0 

0 

0 

0 

0 

0 

0 

DV  .75 

0 

0 

0 

0 

0 

0 

0 

Beta  Weight 

-0.10147 

-0.2664 

-0.50849 

0.60112 

-0.205 

0.08249 

0.51993 

-0.51344 

MISCTOCActivity 

StaffHud 

PERSS3RT0 

PERSALO 

1 

1 

1 

1 

0 

0 

0 

0 

1 

0 

0 

1 

MISCTOCActivity,  PERSALO 

1 

0 

0 

0 

MISCTOCActivity 

0 

0 

0 

1 

PERSALO 

-0.11479 

-0.12006 

0.36351 

-0.0697 
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#  13  -  Tertiary  Task23.  F 

banning 

Variable  intercept 

BOSIntell 

B0Sintel2 

B0SIntel3 

B0Slntel4 

B0SFS9 

B0SMAS6 

B0SADA1 

All  IVs  1.0 

1 

1 

1 

1 

1 

1 

1 

All  IVs  0.0 

0 

0 

0 

0 

0 

0 

0 

DV  .25 

1 

0 

0 

0 

1 

0 

0 

DV  0.5 

1 

1 

0 

0 

0 

0 

0 

DV  .75 

1 

1 

1 

1 

0 

0 

0 

Beta  Weight  -0.06032 

0.64616 

0.59517 

0.66686 

0.76372 

0.2086 

0.66366 

-0.90467 

B0SADA2 

B0SCC1 

BRIntel 

MISCBattleTiming 

MOPPLevel 

StaffHud 

PERSS3RT0 

Humidity 

1 

1 

1 

1 

1 

1 

1 

1 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

1 

1 

0 

1 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

1 

1 

0 

1 

1 

0 

-1 .00819 

0.21674 

-0.6962 

0.5996 

1 .92155 

0.19585 

-0.22018 

-0.24981 

BOSIntell,  BOSFS9,  BRIntel,  MISCBattleTiming,  StaffHud 
BOSIntell,  BOSIntel2  " 

BOSIntell,  BOSIntel2,  BOSIntel3,  BOSIntel4,  BRIntel,  MISCBattleTiming,  StaffHud,  PERSS3RTO 


#  14  -  Missionl.  Pre  Operations  Planning 

Variable  Intercept!  B0SMAS1  B0SMAS2  B0SMAS4  B0SADA1  COMMCS  MISCBattleTiming 


All  IVs  1.0 

1 

1 

1 

1 

1 

1 

All  IVs  0.0 

0 

0 

0 

0 

0 

0 

DV  .25 

1 

0 

0 

0 

0 

0 

DV  0.5 

1 

0 

0 

0 

0 

0 

DV  .75 

1 

1 

0 

1 

1 

0 

Beta  Weight 

0.32216  -0.41072 

0.49275 

0.47737 

-0.61766 

0.39495 

-0.87756 

BOSMAS1 

BOSMAS1 

BOSMAS1 ,  BOSMAS2.BOSADA1 ,  COMMCS 
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#  15  -  Mission2.  Movement  to  Contact 

Variable 

Intercept 

B0SFS2 

B0SMAS1 

B0SCC2 

B0SCC3 

MISCReconOps 

PERSS3N00 

All  IVs  1.0 

1 

1 

1 

1 

1 

1 

All  IVs  0.0 

0 

0 

0 

0 

0 

0 

DV  .25 

1 

1 

0 

1 

0 

1 

DV  0.5 

1 

0 

1 

1 

1 

1 

DV  .75 

1 

1 

1 

1 

1 

1 

Beta  Weight 

0.21417 

-0.07909 

0.08561 

0.22173 

0.2183 

-0.13503 

-0.2145 

B0SFS2,  B0SMAS1,  B0SCC3,  PERSS3NC0 

BOSFS2,BOSCC2,  B0SCC3,  MISCReconOps,  PERSS3NCO 

BOSFS2,  BOSMAS1,  BOSCC2,  BOSCC3,  MISCReconOps,  PERSS3NCO 


#  16  -  Mission3.  Attack 

Variable 

Intercept 

B0SIntel4 

MISCBattleTiming 

MOPPLevel 

All  IVs  1.0 

1 

1 

1 

All  IVs  0.0 

0 

0 

0 

DV  .25 

0 

0 

1 

MOPPLevel 

DV  0.5 

1 

0 

1 

BOSInteW,  MOPPLevel 

DV  .75 

0 

1 

1 

MISCBattleTiming,  MOPPLevel 

Beta  Weight 

-0.15612 

0.35484 

0.41945 

0.1987 

#  17  -  Mission5.  River  Crossing 

Variable 

Intercept 

B0SIntel4 

BOSManl 

B0SADA2 

MISCBattleTiming 

MOPPLevel 

PERSS3NC0 

All  IVs  1.0 

1 

1 

1 

1 

1 

1 

All  IVs  0.0 

0 

0 

0 

0 

0 

0 

DV  .25 

0 

0 

1 

0 

0 

0 

DV  0.5 

1 

0 

0 

1 

1 

1 

DV  .75 

1 

1 

1 

0 

0 

1 

Beta  Weight 

-0.65542 

0.09927 

0.41456 

0.46865 

1 .27217 

-0.72215 

0.11363 

BOSADA2 

BOSInteW,  MISCBattleTiming,  MOPPLevel,  PERSS3NCO 
BOSInteW,  BOSManl,  BOSADA2,  PERSS3NCO 


Appendix  Q  -  Detennination  Of  CoHOST  Simulation  Replication  Count. 

With  stochastic  computer  simulations,  multiple  runs  must  be  conducted  for  each 
combination  of  the  IVs  in  order  to  account  for  the  variability  induced  by  the  random  number 
generation  (Whicker  and  Sigelman,  1991).  When  analysis  is  conducted  on  data  generated  by  a 
simulation  due  to  random  effects  then  the  variance  of  the  output  data  must  be  controlled  so  that  it 
falls  within  a  desired  precision  limit  (Banks  et  ah,  1996).  Stated  succinctly,  a  computer 
simulation  model  involving  Monte  Carlo  detenninations  needs  to  be  repeated  or  “replicated”  as 
many  times  as  necessary  to  get  the  required  precision  (Kelton,  1995).  This  can  be  achieved  by 
making  multiple  replications  of  the  simulation  runs  by  holding  the  IV  levels  constant  and 
changing  the  random  number  seed  at  the  start  of  each  replication  run.  When  a  sufficient  number 
of  replications  have  been  executed  then  the  mean  of  the  output  data  from  the  replications  can  be 
expected  to  fall  within  the  desired  confidence  limit.  For  this  study  it  is  desired  to  have  the  output 
data  from  the  simulation  exhibit  a  95%  probability  of  falling  within  the  confidence  limit  which 
gives  a  specified  error  level,  e,  equal  to  ±.05  of  the  mean.  Following  Banks’  procedures,  (pages 
429-449),  the  required  number  of  replications  can  be  determined  that  needs  to  be  conducted  to 
support  the  intended  analysis. 

An  initial  simulation  run  of  5  replications  was  made  with  a  starting  random  number  seed 
of  1 .  The  model  automatically  used  the  next  random  number  at  the  end  of  each  replication  as  the 
starting  seed  for  the  next  replication.  The  resulting  data  for  workload,  utilization,  tasks  queued, 
tasks  dropped,  and  tasks  interrupted  for  the  Battalion  Commander  is  shown  in  Table  42.  A 
replication  analysis  was  performed  for  each  of  these  DVs  to  detennine  the  number  of  replications 
required  to  satisfy  each  of  these  measures. 


466 


Table  42  -  Data  From  Initial  5  Replication  Simulation  Run 


Workload 

Utilizatio 

Number  of 

Tasks 

Tasks 

Tasks 

Battalion  Commander 

(•Util) 

n 

(•Util) 

Queues 

(*Opdata) 

Intel  i  upted 
(*Opdata) 

S  uspended 
(*Opdata) 

Dropped 

(*Opdata) 

Replication  1 

3561179.34 

92.30% 

50 

90 

ii 

11 

Replication  2 

3549435.31 

92.07% 

48 

90 

ii 

13 

Replication  3 

3556732.44 

92.39% 

54 

90 

10 

11 

Replication  4 

3548795.69 

93.21% 

52 

88 

11 

9 

Replication  5 

3526501.95 

91.95% 

51 

91 

11 

13 

Mean 

3548528.95 

92.38% 

51.00 

89.80 

10.80 

11.40 

Standard  Deviation 

13354.88 

0.0049 

2.24 

1.10 

0.45 

1.67 

5%  Error  Limit  = 

177426.45 

0.0462 

2.55 

4.49 

0.54 

0.57 

05,  relative  to  the  mean 

*  (filename)  ==  name  of  model  output  data  file 


The  desire  is  to  determine  the  number  of  replications  required  so  that  the  relative  error 
(relative  to  the  mean)  for  any  of  the  DVs  does  not  exceed  5  percent.  The  iterative  formula  to 
determine  the  number  of  replications  is  (Banks  et  ah,  1996)  (eq.  12.29,  p.  449)  is  shown  as 
Equation  2. 

'ZaS0 

R>  — ^ — 

£ 

V 

Equation  1  -  Initial  Estimate  for  Number  of  Required  Replications,  R 
where, 

R  =  number  of  replications  required  to  achieve  the  desired  error  level 
Z  =  Z  statistic 

So  =  Standard  Deviation  of  the  computed  parameter  across  the 

simulation  replications 

s  =  Error  Level  Threshold 

a  =  Percent  Error  Level  of  the  mean  value  of  the  computed  parameter 

across  the  simulation  replications. 
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This  expression  is  iteratively  computed  with  the  value  of  the  computed  replications  being 
substituted  for  R  until  the  value  for  R  satisfies  the  greater  than  or  equal  to  condition  at  which 
time  the  value  for  R  becomes  the  required  number  of  replications  necessary  to  compensate  for 
the  random  effects  of  the  simulation. 

Example  Calculation  For  Replication  Analysis  For  Workload  Parameter: 

From  Table  42: 

So  =  1354.88 
s=  117426.45 

a  =.05;  a/2  =  .025;  1 -a/2  =  .975 

.'.  Z  975  =  1.96,  from  Z  table,  page  966  (Winer  et  ah,  1991) 

Thus, 

R>  {(Za/2xS0)/s}2 

R >  {(1.96  x  13354.88)/  177426.45}2=  .02176-1 
So,  use  R  =  1 .  Since  this  is  less  than  50,  use  the  t  distribution,  plug  back  into  the 
formula  and  evaluate.  From  the  t  table,  (Winer  et  ah,  1991),  page  967,  ta/2, i  = 
12.71 

R  ^  {(^97.5,2  X  So)  /  S}“ 

R  >  {(12.71  x  13354.88)  /  177426.45}  2  =  .91524  *  1 

.'.  5  >  1  relationship  is  verified.  As  one  run  is  required  and  5  have  been 

made,  no  additional  runs  are  required  to  satisfy  this  parameter. 

Table  43  shows  the  replication  analysis  for  all  the  DVs.  From  this  analysis  it  is  determined  that 
the  parameter  “Number  of  Queues’  is  the  defining  variable  and  will  require  15  replication  runs  to 
satisfy  the  criteria. 
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Table  43  -  Replication  Analysis  For  Initial  5  Replication  Run 


R>[(Za/2xS0)/s]2 

Battalion 

Commander 

At/2 

f<x/2,df 

So 

e 

R 

R  Adjusted 

Conclusion 

Workload: 

Since  R  <  50,  use  t  disbribution  and  recalculate 

.'.  5  >  1  relationship  is  satisfied.  No  more  runs  required 

Z.975 

^  97.5,1 

So 

8 

R 

R  Adjusted 

1.96 

13354.88 

177426.45 

0.02176 

1 

12.71 

13354.88 

177426.45 

0.91524 

1 

Utilization: 

Since  R  <  50,  use  t  disbribution  and  recalculate 

.•.  5  >  2  relationship  is  satisfied.  No  more  runs  required 

Z.975 

r  97.5,1 

So 

8 

R 

R  Adjusted 

1.96 

0.0049 

0.0462 

0.04395 

1 

12.71 

0.0049 

0.0462 

1.84795 

2 

Number  of  Queues 

Since  R  <  50,  use  t  disbribution  and  recalculate 
.'.  5  >  1 5  ->  No;  Set  R=  1 5  and  reevaluate 

So 

8 

R 

R  Adjusted 

1.96 

1 97.5,2 

2.24 

2.55 

2.95394 

3 

4.3 

2.24 

2.55 

14.21761 

15 

1 97.5,14 

So 

8 

R 

.-.15  >4  ->  Yes,  therefore  Use  R  =  15 

2.14 

2.24 

2.55 

3.52141 

4 

Tasks  Interrupted 

Since  R  <  50,  use  t  disbribution  and  recalculate 
.•.  5  >  10  ->  No;  Set  R=10  and  reevaluate 

Z.975 

So 

8 

R 

R  Adjusted 

1.96 

1 97.5,1 

1.10 

4.49 

0.22867 

1 

12.71 

1.10 

4.49 

9.61567 

10 

f  97.5,9 

So 

8 

R 

.-.  10  >  1  ->  Yes,  therefore  Use  R  =  10 

2.26 

1.10 

4.49 

0.30402 

1 

Tasks  Suspended 

Since  R  <  50,  use  t  disbribution  and  recalculate 

5  >  13  ->  No;  Set  R=13  and  reevaluate 

Z.975 

So 

8 

R 

R  Adjusted 

1.96 

1 97.5,2 

0.45 

0.54 

2.63484 

3 

4.3 

0.45 

0.54 

12.68176 

13 

1 97.5,12 

So 

8 

R 

.'.  13  >  4  ->  Yes,  therefore  Use  R  =  13 

2.18 

0.45 

0.54 

3.25953 

4 

469 


This  page  left  intentionally  blank. 


470 


Vita 


Sam  E.  Middlebrooks,  a  career  Federal  Civil  Service  employee  for  over  20  years,  is  a 
member  of  the  Human  Research  and  Engineering  Directorate  (HRED)  of  the  U.S.  Army 
Research  Laboratory  (ARL)  where  he  is  the  Chief  of  the  ARL-HRED  Field  Element  at  Fort 
Hood,  Texas.  Previously,  through  17  years  of  Federal  service  in  progressive  assignments  in  the 
Federal  Republic  of  Germany,  he  was  chief  of  the  programming  team  at  a  joint  U.S.  Anny  and 
U.S.  Air  Force  computer  wargaming  center  in  Germany  and  later  became  the  director  of  the 
largest  corps  level  battle  simulation  center  in  Europe.  Sam  holds  a  Bachelor  of  Science  degree  in 
Mechanical  Engineering  from  the  University  of  Texas  at  Arlington  and  a  Master  of  Science 
degree  in  Industrial  and  Systems  Engineering  (Human  Factors  option)  from  the  Virginia 
Polytechnic  Institute  and  State  University  in  Blacksburg,  Virginia.  He  is  a  retired  U.S.  Army 
Annor  officer  with  operational  experience  in  annored  and  cavalry  units  from  platoon  through 
corps  levels.  After  two  years  of  full  time  study  on  campus  at  Virginia  Tech,  beginning  in  the 
Fall  of  1999,  Sam  is  completing  work  on  his  Ph.D.  with  a  dissertation  project  that  has  the  long 
tenn  goal  of  developing  a  new  research  program  at  Fort  Hood  conducting  modeling  and 
simulation  of  human  cognitive  performance  in  military  command  and  control  teams. 

Membership  in  professional  societies  include  the  Human  Factors  and  Ergonomics  Society 
(HFES),  the  Military  Operations  Research  Society  (MORS),  the  Association  for  Computing 
Machinery  (ACM),  and  the  Alpha  Pi  Mu  Industrial  Engineering  Honor  Society. 


471 


NO.  OF 
COPIES 


ORGANIZATION 


NO.  OF 

COPIES  ORGANIZATION 


1  DEFENSE  TECHNICAL 
(PDF  INFORMATION  CTR 
ONLY)  DTICOCA 

8725  JOHN  J  KINGMAN  RD 
STE  0944 

FORT  BEL  VOIR  VA  22060-6218 

1  US  ARMY  RSRCH  DEV  &  ENGRG  CMD 
SYSTEMS  OF  SYSTEMS 
INTEGRATION 
AMSRD  SS  T 
6000  6TH  ST  STE  100 
FORT  BEL  VOIR  VA  22060-5608 

1  INST  FOR  ADVNCD  TCFFNLGY 
THE  UNIV  OF  TEXAS  AT  AUSTIN 
3925  W  BRAKER  LN  STE  400 
AUSTIN  TX  78759-5316 

1  DIRECTOR 

US  ARMY  RESEARCH  LAB 
IMNE  ALC  IMS 
2800  POWDER  MILL  RD 
ADELPHI  MD  20783-1197 

1  DIRECTOR 

US  ARMY  RESEARCH  LAB 
AMSRD  ARL  Cl  OK  TL 
2800  POWDER  MILL  RD 
ADELPHI  MD  20783-1197 

2  DIRECTOR 

US  ARMY  RESEARCH  LAB 
AMSRD  ARL  CS  OK  T 
2800  POWDER  MILL  RD 
ADELPHI  MD  20783-1197 

1  DIRECTOR 

US  ARMY  RESEARCH  LAB 
AMSRD  ARL  D 
2800  POWDER  MILL  RD 
ADELPHI  MD  20783-1 197 

1  ARMY  RSCH  LABORATORY  -  HRED 
ATTN  AMSRD  ARL  HR  MN  R  SPENCER 
DCSFDI  HF 

HQ  USASOC  BLDG  E2929 
FORT  BRAGG  NC  28310-5000 

1  ARMY  RSCH  LABORATORY  -  HRED 
ATTN  AMSRD  ARL  HR  MA  J  MARTIN 
MYER  CENTER  RM2D311 
FT  MONMOUTH  NJ  07703-5630 


1  ARMY  RSCH  LABORATORY  -  HRED 
ATTN  AMSRD  ARL  HR  S  DRMSTRUB 
6359  WALKER  LANE  SUITE  100 
ALEXANDRIA  VA  22310 

1  ARMY  RSCH  LABORATORY  -  HRED 

ATTN  AMSRD  ARL  HR  MV  S  MIDDLEBROOKS 
HQ  USAOTC 

91012  STATION  AVE  ROOM  1 1 1 
FT  HOOD  TX  76544-5073 

1  ARMY  G1  DAPE  MR  B  KNAPP 

300  ARMY  PENTAGON  ROOM  2C489 
WASHINGTON  DC  20310-0300 

ABERDEEN  PROVING  GROUND 

2  DIRECTOR 

US  ARMY  RSCH  LABORATORY 

ATTN  AMSRD  ARL  Cl  OK  TP  S  FOPPIANO 

BLDG  459 

1  DIRECTOR 

US  ARMY  RSCH  LABORATORY 

ATTN  AMSRD  ARL  HR  M  F  PARAGALLO 

BLDG  459 

1  DIRECTOR 

US  ARMY  RSCH  LABORATORY 
ATTN  AMSRD  ARL  HR  S  L  PIERCE 
BLDG  459 

1  DIRECTOR 

US  ARMY  RSCH  LABORATORY 
ATTN  AMSRD  ARL  HR  SE  D  HEADLEY 
BLDG  459 


472 


